Parallel computer system with physically separate tree networks for data and control messages

ABSTRACT

A digital computer comprises a plurality of processing elements, a communications router, and a control network. Each processing element performs data processing operations in connection with commands, at least some of the processing elements performing the data processing operations in connection with the commands in messages they receive over the control network. Each processing element also generates and receives data transfer messages, each including an address portion containing an address, for transfer to another processing element as identified by the address. At least one of the processing elements further generates the control network messages for transfer over the communications router. The communications router comprises router nodes interconnected in the form of a &#34;fat-tree,&#34; and the control network comprises control network nodes interconnected in the form of a tree, with the processing elements being connected at the leaf nodes of the respective communications router and control network.

This application is a division of application Ser. No. 08/183,219, filedJan. 14, 1994, now U.S. Pat. No. 5,388,214 which is a continuation ofapplication Ser. No. 07/592,029, filed Oct. 3, 1990, now abandoned.

INCORPORATION BY REFERENCE

Guy E. Blelloch, Scan Primitives and Parallel Vector Models, (Ph.D.Dissertation, Massachusetts Institute of Technology: 1988), incorporatedherein by reference.

U.S. patent appn. Ser. No. 07/489,079, filed Mar. 5, 1990, now U.S. Pat.No. 5,118,975 in the name of W. Daniel Hillis, et al., entitled DigitalClock Buffer Circuit Providing Controllable Delay, and assigned to theassignee of the present application, incorporated herein by reference.

FIELD OF THE INVENTION

The invention relates generally to the field of digital computersystems, and more particularly to massively parallel computing systems.The invention particularly provides arrangements for controllingprocessors in a computing system having a large number of processors,for facilitating transfer of data among the processors and forfacilitating diagnosis of faulty components in the computing system.

BACKGROUND OF THE INVENTION

A digital computer system generally comprises three basic elements,namely, a memory element, an input/output element and a processorelement. The memory element stores information in addressable storagelocations. This information includes data and instructions forprocessing the data. The processor element fetches information from thememory element, interprets the information as either an instruction ordata, processes the data in accordance with the instructions, andreturns the processed data to the memory element. The input/outputelement, under control of the processor element, also communicates withthe memory element to transfer information, including instructions andthe data to be processed, to the memory, and to obtain processed datafrom the memory.

Most modern computing systems are considered "von Neumann" machines,since they are generally constructed according to a paradigm attributedto John von Neumann. Von Neumann machines are characterized by having aprocessing element, a global memory which stores all information in thesystem, and a program counter that identifies the location in the globalmemory of the instruction being executed. The processing elementexecutes one instruction at a time, that is, the instruction identifiedby the program counter. When the instruction is executed, the programcounter is advanced to identify the location of the next instruction tobe processed. (In many modern systems, the program counter is actuallyadvanced before the processor has finished processing the currentinstruction.)

Von Neumann systems are conceptually uncomplicated to design andprogram, since they do only one operation at a time. A number ofadvancements have been made to the original von Neumann paradigm topermit the various parts of the system, most notably the variouscomponents of the processor, to operate relatively independently andachieve a significant increase in processing speed. One such advancementis pipelining of the various steps in executing an instruction,including instruction fetch, operation code decode (a typicalinstruction includes an operation code which identifies the operation tobe performed, and in most cases one or more operand specifiers, whichidentify the location in memory of the operands, or data, to be used inexecuting the instruction), operand fetch, execution (that is,performing the operation set forth in the operation code on the fetchedoperands), and storing of processed data, which steps are performedrelatively independently by separate hardware in the processor. In apipelined processor, the processor's instruction fetch hardware may befetching one instruction while other hardware is decoding the operationcode of another instruction, fetching the operands of still anotherinstruction, executing yet another instruction, and storing theprocessed data of a fifth instruction. Since the five steps areperformed sequentially, pipelining does not speed up processing of anindividual instruction. However, since the processor begins processingof additional instructions before it has finished processing a currentinstruction, it can speed up processing of a series of instructions.

A pipelined processor is obviously much more complicated than a simpleprocessor in a von Neumann system, as it requires not only the variouscircuits to perform each of the operations (in a simple von Neumannprocessor, many circuits could be used to perform several operations),but also control circuits to coordinate the activities of the variousoperational circuits. However, the speed-up of the system can bedramatic.

More recently, some processors have been provided with executionhardware which includes multiple functional units each being optimizedto perform a certain type of mathematical operation. For example, someprocessors have separate functional units for performing integerarithmetic and floating point arithmetic, since they are processed verydifferently. Some processors have separate hardware functional unitseach of which performs one or only several types of mathematicaloperations, including addition, multiplication, and division operations,and other operations such as branch control and logical operations, allof which can be operating concurrently. This can be helpful in speedingup certain computations, most particularly those in which severalfunctional units may be used concurrently for performing parts of asingle computation.

In a von Neumann processor, including those which incorporate pipeliningor multiple functional units (or both, since both may be incorporatedinto a single processor), a single instruction stream operates on asingle data stream. That is, each instruction operates on data to enableone calculation at a time. Such processors have been termed "SISD," forsingle-instruction/single-data." If a program requires a segment of aprogram to be used to operate on a number of diverse elements of data toproduce a number of calculations, the program causes the processor toloop through that segment for each calculation. In some cases, in whichthe program segment is short or there are only a few data elements, thetime required to perform such a calculation may not be unduly long.

However, for many types of such programs, SISD processors would requirea very long time to perform all of the calculations required.Accordingly, processors have been developed which incorporate a largenumber of processing elements all of which may operate concurrently onthe same instruction stream, but with each processing element processinga separate data stream. These processors have been termed "SIMD"processors, for "single-instruction/multiple-data."

SIMD processors are useful in a number of applications, such as imageprocessing, signal processing, artificial intelligence, databaseoperations, and computer simulation of a number of things, such aselectronic circuits and fluid dynamics. In image processing, eachprocessing element may be used to perform processing on a pixel("picture element") of the image to enhance the overall image. In signalprocessing, the processors concurrently perform a number of thecalculations required to perform such computations as the "Fast Fouriertransform" of the data defining the signal. In artificial intelligence,the processors perform searches on extensive rule bases representing thestored knowledge of the particular application. Similarly, in databaseoperations, the processors perform searches on the data in the database,and may also perform sorting and other operations. In computersimulation of, for example, electronic circuits, each processor mayrepresent one part of the circuit, and the processor's iterativecomputations indicate the response of the part to signals from otherparts of the circuit. Similarly, in simulating fluid dynamics, which canbe useful in a number of applications such as weather predication andairplane design, each processor is associated with one point in space,and the calculations provide information about various factors such asfluid flow, temperature, pressure and so forth.

Typical SIMD systems include a SIMD array, which includes the array ofprocessing elements and a router network, a control processor and aninput/output component. The input/output component, under control of thecontrol processor, enables data to be transferred into the array forprocessing and receives processed data from the array for storage,display, and so forth. The control processor also controls the SIMDarray, iteratively broadcasting instructions to the processing elementsfor execution in parallel. The router network enables the processingelements to communicate the results of a calculation to other processingelements for use in future calculations.

Several routing networks have been used in SIMD arrays and others havebeen proposed. In one routing network, the processing elements areinterconnected in a matrix, or mesh, arrangement. In such anarrangement, each processing element is connected to, and communicateswith, four "nearest neighbors" to form rows and columns defining themesh. This arrangement can be somewhat slow if processing elements needto communicate among themselves at random. However, the arrangement isinexpensive and conceptually simple, and may suffice for some types ofprocessing, most notably image processing. The "Massively ParallelProcessor" manufactured by Goodyear Aerospace Corporation is an exampleof a SIMD array having such a routing network.

In another routing network, processing elements are interconnected in acube or hypercube arrangement, having a selected number of dimensions,for transferring data, in the form of messages, among the processingelements. The arrangement is a "cube" if it only has three dimensions,and a "hypercube" if it has more than three dimensions. U.S. Pat. No.4,598,400, entitled Method and Apparatus For Routing Message Packets,issued Jul. 1, 1986 to W. Daniel Hillis, and assigned to the assignee ofthe present application, describes a system having a hypercube routingnetwork. In the system described in the '400 patent, multiple processingelements are connected to a single routing node, and the routing nodesare interconnected in the hypercube.

Another routing arrangement which has been proposed is a crossbarswitch, through which each processing element can communicate directlywith any of the other processing elements. The crossbar switch providesthe most efficient communications of any of the routing networksproposed. However, a crossbar switch also has the most connections andswitching elements, and thus is the most expensive and also the mostsusceptible to failure due to broken connections and faulty switchingelements. Thus, crossbar switch arrangements are rarely used, exceptwhen the number of processing elements is fairly small, since thecomplexity of a crossbar switch increases with the square of the numberof processing elements.

Yet another routing arrangement is an omega network, in which switchingis performed through a number of serially-connected stages. Each stagehas two inputs, each connected to the outputs of a prior stage orprocessing elements, has two outputs which may be connected to theinputs of a subsequent stage or processing elements. The "Butterfly"computer system manufactured by Bolt Beranek & Newman uses such anetwork.

SUMMARY OF THE INVENTION

The invention provides a new and improved parallel computer system.

In brief summary, the new computer includes a plurality of processingelements, a command processor, a diagnostic processor and acommunications network. The processing elements each performs dataprocessing and data communications operations in connection withcommands. The processing elements also performing diagnostic operationsin response to diagnostic operation requests and providing diagnosticresults in response thereto. The command processor generates commandsfor the processing elements, and also performs diagnostic operations inresponse to diagnostic operation requests and providing diagnosticresults in response thereto. The diagnostic processor generatesdiagnostic requests. The communication network includes three elements,including a data router, a control network and a diagnostic network. Thedata router is connected to the processing elements for facilitating thetransfer of data among them during a data communications operation. Thecontrol network is connected to the processing elements and the commandprocessor for transferring commands from the command processor to theprocessing elements. The diagnostic network connected to the processingelements, the command processor and the diagnostic processor fortransferring diagnostic requests from the diagnostic processor to theprocessing elements and the command processor and for transferringdiagnostic results from the processing elements and the commandprocessor to the diagnostic processor.

BRIEF DESCRIPTION OF THE DRAWINGS

This invention is pointed out with particularity in the appended claims.The above and further advantages of this invention may be betterunderstood by referring to the following description taken inconjunction with the accompanying drawings, in which:

FIG. 1 is a general block diagram of a massively parallel computersystem constructed in accordance with the invention;

FIGS. 2A and 2B are block diagrams useful in understanding the structureand operation of the data router of the computer system of FIG. 1;

FIG. 3 is a diagram depicting the structure of message packetstransferred over the data router;

FIGS. 4A and 4B are block diagrams useful in understanding the structureand operation of the control network of the computer system of FIG. 1;

FIG. 5 is a diagram depicting the structure of message packetstransferred over the control network;

FIGS. 6A through 6C are block diagrams useful in understanding thestructure and operation of the diagnostic network of the computer systemof FIG. 1;

FIG. 7 is a diagram depicting the structure of message packetstransferred over the diagnostic network;

FIG. 8 is a general block diagram of a processing element in thecomputer system depicted in FIG. 1;

FIG. 9A-1 comprises a general block diagram of a data router interfacecircuit useful in interfacing the processing element depicted in FIG. 8to the data router of the computer system depicted in FIG. 1, FIGS.9A-2A and 9A-2B contain definitions of registers in the data routerinterface and FIGS. 9B-1 through 9D-7 comprise logic diagrams of thedata router interface;

FIG. 10A comprises a general block diagram of a control networkinterface circuit useful in interfacing the processing element depictedin FIG. 8 to the control network of the computer system depicted in FIG.1, FIGS. 10A-1 contains a definitions of a register in the controlnetwork interface and FIGS. 10B through 10G comprise logic diagrams ofthe control network interface;

FIG. 11A is a general block diagram of a data router node used in thedata router described in connection with FIGS. 2A and 2B, and FIGS.11B-1 through 11D comprise detailed block and logic diagrams of the datarouter node;

FIG. 12A is a general block diagram of a control network node used inthe control network described in connection with FIGS. 4A and 4B, andFIGS. 12B-1 through 12D-1 comprise detailed block and logic diagrams ofthe control router mode; and

FIG. 13A is a general block diagram of a diagnostic network node used inthe diagnostic network described in connection with FIG. 6, and FIGS.13B-1 through 13C comprise detailed block and logic diagrams of thediagnostic network node.

DETAILED DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT I. GeneralDescription

A. General Description of Computer System

FIG. 1 is a general block diagram of a massively parallel computersystem 10 constructed in accordance with the invention. With referenceto FIG. 1, system 10 includes a plurality of processing elements 11(0)through 11(N) (generally identified by reference numeral 11), scalarprocessors 12(0) through 12(M) (generally identified by referencenumeral 12) and input/output processors 13(0) through 13(K) (generallyidentified by reference numeral 13). Input/output units (not shown),such as, for example, disk and tape storage units, video displaydevices, printers and so forth may be connected to the input/outputprocessors to supply information, including data and program commands,for processing by the processing elements 11 and scalar processors 12 inthe system, and may also receive processed data for storage, display andprinting. The scalar processors 12 may also be connected to input/outputunits including, for example, video display terminals which permit oneor more operators to generally control system 10.

The system 10 further includes a control network 14, a data router 15and a diagnostic network 16. The control network 14 permits one or morescalar processors 12 to broadcast program commands to the processingelements 11. The processing elements 11 execute the commands generallyconcurrently. The control network 14 also permit the processing elements11 to transfer status information to the scalar processors 12. Thecontrol network 14 is also used by the processing elements 11 to performselected types of arithmetic operations, termed "scan" and "reduce"operations, as described below. The control network 14 may also be usedto provide synchronization among the processing elements 11.

The data router 15 transfers data among the processing elements 11,scalar processors 12 and input/output processors 13. In particular,under control of the scalar processors 12, the input/output processors13 retrieve data to be processed from the input/output units anddistributes it to the respective scalar processors 12 and processingelements 11. During processing, the scalar processors 12 and processingelements 11 can transfer data among themselves over the data router 15.In addition, the processing elements 11 and scalar processors 12 cantransfer processed data to the input/output processors 13. Under controlof the scalar processors 12, the input/output processors 13 can directthe processed data that they receive from the data router 15 toparticular ones of the input/output units for storage, display,printing, or the like.

The diagnostic network 16, under control of a diagnostic processor (notshown), facilitates testing of other portions of the system 10 toidentify, locate and diagnose defects. The diagnostic processor maycomprise one or more of the scalar processors 12. In addition, thediagnostic network 16 may be used to establish selected operatingconditions in the other portions of the system 10 as described below.

The system 10 is synchronous, that is, all of its elements operate inaccordance with a global SYS CLK system clock signal provided by a clockcircuit 17.

One particular embodiment of system 10 may include hundreds or manythousands of processing elements 11 operating on a single problem inparallel under control of commands broadcast to them by the scalarprocessors 12. In that embodiment, the processing elements 11 operate inparallel on the same command on their individual sets of data, therebyforming a parallel computer system. In addition, the system 10 may bedynamically logically partitioned, as described below, into multiplesubsystems which may concurrently operate on separate problems orseparate parts of a single problem. In that case, each partitionincludes at least one scalar processor 12 and a plurality of processingelements 11.

B. General Description of Communications Networks

1. Data Router 15

Before proceeding to a detailed description of the system 10 and itsvarious components, it would be helpful to generally describe thestructures of the control network 14 and data router 15. The data router15 and control network 14 both transfer information in the form ofmessage packets, which will be described in detail below in connectionwith FIGS. 3 and 5, respectively. FIGS. 2A and 2B depict the generalstructure of the data router 15 and FIGS. 4A and 4B depict the generalstructure of the control network 14.

With reference to FIG. 2A, the data router 15 is generallytree-structured, having a plurality of data router node groups 20(i,j)("i" and "j" are integers) organized in a plurality of levels eachidentified by the index "i" in reference numeral 20(i,j). A data routernode group 20(i,j) at each level "i" is connected to a selected numberof data router node groups 20(i-1,j) in the next lower level "i-1" toform a tree. As will be described in detail below, the data router nodegroups 20(i,j) perform message switching operations to transfer data, inthe form of data router message packets, among the processing elements11, scalar processors 12 and input/output processors 13, which arecollectively identified as leaves 21(0) through 21(N) (generallyidentified by reference numeral 21). Each data router node group 20(1,j)in the lowest level is connected to one or more leaves 21. In thereference numeral 20(i,j), the index (j) uniquely identifies each of thedata router node groups 20(i,j) at each level "i."

In the data router 15 represented in FIG. 2A, the data router node group20(M,0) at the highest level "M" is termed the "physical root" of thetree. At each level "i", each data router node group 20(i,j) is termedthe "parent" of data router node groups 20(i-1,j) connected thereto, andeach data router node group 20(i-1,j) is termed a "child" of the datarouter node group 20(i,j) to which it is connected. It will beappreciated that the data router node group 20(i,j) will also be a childof the data router node group 20(i+1,j) connected thereto. In oneparticular embodiment, each data router node group 20(i,j) in aparticular level "i" is connected to four child data router node groups20(i-1,j); in that embodiment, the "fan-out" of the tree, that is, thenumber of children connected to each parent, is four. It will beappreciated from the following that the fan-out need not be constant,but may vary from level to level and also among data router node groups20(i,j) within the same level.

The structure of the data router 15 is further termed a "fat-tree", andwill be particularly described in connection with FIG. 2B. Withreference to FIG. 2B, at least some of the data router node groups20(i,j) includes at least one, and typically two or more data routernodes 22(i,j,k), wherein "k" is an integer that uniquely identifies eachdata router node within a data router node group 20(i,j). Each datarouter node 22(i,j,k) in a data router node group 20(i,j) is connectedto a plurality of data router nodes 22(i+1,j,k) in level "i+1," with theconnections being established so that the data router nodes 22(i,j,k) ineach data router node group 20(i,j) are connected to different ones ofthe data router nodes 22(i+1,j,k) in the data router node group 20(i,j)in level "i+1." For example, in data router node group 20(1,0), datarouter node 22(1,0,0) is connected to data router nodes 22(2,0,0) and22(2,0,1) of data router node group 20(2,0), and data router node22(1,0,1) is connected to data router nodes 22(2,0,2) and 22(2,0,3) ofdata router node group 20(2,0).

In addition, each data router node 22(i,j,k) in a parent data routernode group 20(i,j) is connected to one data router node 22(i-1,j,k) inthat parent's child data router node groups 20(i-1,j). Accordingly, asshown in FIG. 2B, data router node (2,0,0) in data router node group20(2,1) is connected to one data router node 22(1,j,0), where "j" equals0, 1, 2 and 3, in each of the data router node groups 20(1,0) through21(1,3).

It will be appreciated that the collection of data router nodes22(i,j,k) from each leaf 21 to and including the data router nodes22(m,0,k) in the root data router node group 20(M,0) essentially formsan inverted tree. Each leaf 21 effectively comprises the root of oneinverted tree and the data router nodes 22(M,0,k) of the root datarouter node group 20(M,0) form all of the leaves of all of the invertedtrees defined by the collection of leaves 21. The number of data routernodes 22(i,j,k) in each data router node group 20(i,j) at a particularlevel "i" in the tree defining data router 15 will be determined by thefan-out at each level from level "1" to level "i" in the inverted tree.The fan-out at a particular level "i" is the number of data router nodes22(i+1,j,k) at level "i+1" to which each data router node 22(i,j,k) atlevel "i" is connected. Thus, for example, since data router node22(1,0,0) of data router node group 20(1,0) in level "1" is connected totwo data router nodes 22(2,0,0) and 22(2,0,1) of data router node groups20(2,0) in level "2," the fan-out from data router node 22(1,0,0) istwo. In one particular embodiment, the fan-out from data router nodes22(i,j,k) at a particular level "i" is the same for the entire level,but it may differ from level to level as described below.

As noted above, the data router 15 transfers message packets among theprocessing elements 11, scalar processors 12 and input/output processors13, all of which are represented by leaves 21. Each connection shown inFIG. 2B between a leaf 21 and a data router node 22(1,j,k) of level 1,which is represented by a line therebetween, actually represents twounidirectional data paths, one for transferring a message packet in eachdirection. Thus, for example, the connection between leaf 21(0) and datarouter node 22(1,0,0) of data router node group 20(1,0) represents twodata paths. One data path is used by the leaf 21(0) to transmit amessage packet to the data router node 22(1,0,0) for delivery to anotherleaf 21(x). The other data path is used by the data router node22(1,0,0) to deliver message packets originating at other leaves 21destined for the leaf 21(0).

Similarly, each connection between a data router node 22(i,j,k) of alevel "i" and a data router node 22(i+1,j,k) of a level "i+1," which isalso represented in FIG. 2B by a line, represents two unidirectionaldata paths, one for transferring a message packet in each direction.Thus, for example, the connection between data router node 22(1,0,0) ofdata router node group 20(1,0) and data router node 22(2,0,0) representstwo data paths, one used to transfer message packets from data routernode 22(1,0,0) to data router node 22(2,0,0) and the other to transfermessage packets in the opposite direction, that is, from data routernode 22(2,0,0) to data router node 22(1,0,0).

Transfer of a message packet from one leaf 21(x) to another leaf 21(y)through the data router 15 message transfer proceeds in two generaloperations. First, the data router nodes 22(i,j,k) transfer the messagepacket first "up the tree," that is, to data router nodes insuccessively higher levels, until it reaches a selected maximum leveldetermined in part by the separation between the source and destinationleaves. After a message packet has reached the selected maximum level,the transfer continues "down the tree", during which the data routernodes 22(i,j,k) transfer the message packet to data router nodes atsuccessively lower levels until it is delivered to the destination leaf21(y). As will be clear from the detailed description of the structureand operation of a data router node 22(i,j,k) in FIGS. 11A through 11Dbelow, the data router 15 can transfer a plurality of messagesconcurrently, any of the data router nodes 22(i,j,k) can direct messagesup the tree and other messages down the tree at the same time.

Before proceeding further, it may be helpful to describe the structureof a message packet transferred over the data router 15. With referenceto FIG. 3, a data router message packet 30 includes three generalportions, including a message address portion 31, a message data portion32, and a checksum portion 33, each comprising one or more "flits." Inone embodiment, each flit comprises four bits, which are transferred inparallel over a data router connection, that is, between a leaf 21 and adata router node 22(i,j,k) or between two data router nodes 22(i,j,k).

The message data portion 32 includes several elements, including alength flit 34, a tag flit 35 and one or more data flits 36(0) through36(N) (generally identified by reference numeral 36). The data flits 36generally contain the actual message data being transferred over thedata router 15, which may vary from packet to packet. The tag flit 35contains control information which may be used by the destination leaf,identified herein by reference numeral 22(y), in processing the data.The contents of the length flit 34 are identify the number of flits inthe message data portion 32, and may vary depending on the amount ofdata being transferred in a particular packet. In one particularembodiment, the contents of length flit 34 identify the number ofthirty-two bit words in the data flits 36 of the message packet. In thatembodiment, the number of data flits 36 in the message packet is eighttimes the value in the length flit 34.

The checksum portion 33 contains a value which is used in detectingerrors in packet transmission over the data router 15.

The data router 15 uses the contents of the message address portion 31to determine the path to be traversed by the message packet 30 from thesource leaf to the destination leaf. The message address portion 31includes a header 40, which identifies the selected maximum level towhich the message packet is to be transferred when going up the tree,and a down path identification portion 41 which identifies the path downthe tree to the destination leaf 21(y) when going down the tree. Whendirecting a message packet up the tree, a data router node 22(i,j,k) atlevel "i," randomly selects one of the data router nodes 22(i+1,j,k)connected thereto in level "i+1" in data router node group 20(i+1,j) toreceive the message packet. Other than specifying the selected maximumheight for the message packet, the packet does not otherwise specify theparticular path it is to take up the tree.

The down path identification portion 41 of message packet 30 defines thepath the packet is to take down the tree from the data router node group20(i,j) at the selected maximum level to the destination leaf 21(y). Thedown path identification portion includes one or more down pathidentifier fields 42(1) through 42(M) (generally identified by referencenumeral 42). The successive down path identifier fields 42, beginningwith field 42(M), are used by the data router nodes 22(i,j,k) atsuccessively lower levels as they direct the packet downwardly in thetree.

The down path identifier field 42(i) for level "i" identifies the childdata router node group 20(i-1,j) to which the parent data router nodegroup 20(i,j) that receives the packet at level "i" is to direct themessage packet 30. It will be appreciated that the down path identifierfields 42 need not specifically identify one of the data router nodes22(i-1,j,k) in the data router node group 20(i,j) at each level to whichthe message packet is to be directed, since the path down the tree iseffectively a traversal of the inverted tree of which the destinationleaf 21(y) is the root.

In one embodiment, in which each parent data router node group 20(i,j)is connected to four child data router node groups 20(i-1,j) or fourleaves 21, each down path identifier field 42 comprises two bits thatare binary encoded to identify one of the four children to which themessage is to be directed. As indicated by FIG. 3, two fields 42 arepacked into a single four-bit flit in the message packet 30. Since onedown path identifier field 42 is used to at each level (i) in thedownward traversal, the number of down path identifier fields 42required to define the downward path corresponds to the selected maximumlevel in the path up the tree, which, in turn, corresponds to thecontents of header 40. During the downward traversal mode, the datarouter nodes 22(i,j,k) through which a message packet 30 passesdecrement the contents of the header 40 and, after both down pathidentifier fields 42 contained in a flit have been used, discard theflit. Thus, the length and content of a message packet 30 may change asit is being passed down the tree.

It will be appreciated that the addressing arrangement provided by theheader 40 and down path identification portion 41 can be viewed asfollows. The selected maximum height in header 40 effectively identifiesthe data router node group 20(i,j) which is the root of a sub-tree,preferably the smallest sub-tree, of the data router 15 that containsboth the source leaf 21(x) and the destination leaf 21(y). On the otherhand, the down path identification portion 41 details the exact pathfrom that root to the destination leaf 21(y).

The provision of increasing numbers of data router nodes 22(i,j,k) indata router node groups 20(i,j) at higher levels in the data router 15,thereby resulting in a "fat-tree" design, provides several advantages.In a massively parallel computer SIMD system, processing elements 11typically transfer messages during a message transfer operation,initiated by commands from the scalar processors 12. During a messagetransfer operation, a large number of processing elements 11 maytransfer messages concurrently. If the data router 15 did not haveincreasing numbers of data router nodes 22(i,j,k) at higher levels towhich the message packets 30 can be directed when going up the tree, thebandwidth of the data router 15, that is, the rate at which it cantransfer message packets 30, would decrease at higher levels.

Since increasing numbers of data router nodes 22(i,j,k) are provided athigher levels in the "fat-tree" design, the reduction in bandwidth athigher levels can be minimized or controlled. As noted above, thefan-out of data router node groups 20(i,j), that is, the number of datarouter nodes 22(i+1,j,k) at level "i+1" connected to each data routernode 22(i,j,k) at level "i" can vary from level to level, and can beselected to maintain a desired minimum bandwidth between the respectivelevels "i" and "i+1." Alternatively, the fan-outs from each level to thenext higher level can be selected so that the entire data router 15 hasa selected minimum bandwidth.

Further, as noted above, each data router node 22(i,j,k) randomlyselects the data router node 22(i+1,j,k) in the next higher level towhich it directs a message packet 30 in the path up the tree.Accordingly, the message packets are randomly distributed through thehigher levels of the tree, which minimizes the likelihood of bottlenecksand maximizes the bandwidth in the higher levels.

As shown in FIGS. 2A and 2B, each data router node group 20(i,j), and inparticular each data router node 22(i,j,k), in the data router 15receives an AFD(i,j) all-fall-down (i,j) signal. The AFD(i,j)all-fall-down (i,j) signal is provided by the control network 14, aswill be described below in connection with FIGS. 4A and 4B, undercontrol of the scalar processors 12 to initiate a context switchoperation. The AFD(i,j) all-fall-down (i,j) signal, when asserted,enables the data router 15 to enter an all-fall-down mode, in which itquickly empties itself of message packets. In response to the AFD(i,j)all-fall-down (i,j) signal, the data router 15 directs all messagepackets 30 directly down the tree to the leaves 21, where they arestored until the context in which the message packets were generated isrestored. At that point, the leaves 21 which receive such messages cantransmit them over the data router 15, which will deliver them to theintended destinations.

In contrast to normal operation described above, in which the contentsof the header 40 are decremented and flits containing down pathidentifier fields 42 discarded as the message packet 30 is directed downthe tree, when the AFD(i,j) all-fall-down (i,j) signal is asserted thecontents of the header 40 are not decremented and no changes are made tothe flits containing the down path identifier fields 42. When thecontext is restored and the leaves 21 return the message packets to thedata router 15, they will be delivered to the proper destination leaves.This can be seen from the following explanation.

In the following explanation, reference numerals 21(x) and 21(y) willrefer to the original source and destination leaves, respectively, for amessage packet 30 and reference numeral 21(x') will refer to theintermediate storage leaf which receives and stores the message packet30 while the context in which the data router message packet 30 wasgenerated is being switched out. First, for those message packets thatare being transferred up the tree or that have reached the selectedmaximum height when the AFD(i,j) all-fall-down (i,j) signal is asserted,the contents of the header 40 and down path identification portion 41are the same as when they were originally transmitted by the source leaf21(x). Since the intermediate storage leaf 21(x') receives the messagepacket 30 it must be part of a sub-tree of the data router 15 thatincludes both the source leaf 21(x) and the destination leaf 21(y).Further, the sub-tree has the same root data router node group 20(i,j)that the message packet 30 would have reached had the AFD(i,j)all-fall-down (i,j) signal not been asserted. Accordingly, when theintermediate storage leaf 21(x') transmits the message packet over thedata router 15, the packet will go up the tree and reach the same datarouter node group 20(i,j) that it would have reached if the AFD(i,j)all-fall-down (i,j) signal had not been asserted, and from there willfollow the same downward path, defined by the down path identificationportion 41, that it would have taken.

On the other hand, if a message packet is being transferred down thetree when the AFD(i,j) all-fall-down (i,j) signal is asserted, prior tothe signal's assertion the contents of the header field 40 aredecremented as the message packet is passed from level to level.Accordingly, it will be appreciated that, when the message packet 30 istransmitted by the intermediate storage leaf 21(x'), in its path up thetree it will go only to a data router node group 20(i,j) at the levelindicated in the header field 40, which, in turn, corresponds to thedata router node group 20(i,j) which controlled the direction oftransfer of the message packet 30 when the AFD(i,j) all-fall-down (i,j)signal signal was asserted. It will be appreciated that the data routernode group 20(i,j) that the message packet 30 reaches may not be theroot of a sub-tree that includes the source leaf 21(x). However, it willbe the root of a sub-tree that includes both the intermediate storageleaf 21(x'), since the message packet 30 was transferred from that datarouter node group 20(i,j) to the intermediate storage leaf 21(x'), andthe destination leaf 21(y), since the message packet 30 could have beentransferred from that data router node group 20(i,j) to the destinationleaf had the AFD all-fall-down (i,j) signal not been asserted.

As will be described in further detail below, each leaf 21 maintains amessage counter that it increments when it tranmsits a message packetover the data router 15, and that it decrements when it receives amessage packet from the data router 15. As noted above, the controlnetwork 14 performs selected arithmetic operations, whose results can beprovided to the processing elements 11 and scalar processors 12. Byenabling the control network 14 to perform selected arithmeticoperations using the values of the message counters, the results canidentify when all of the message packets that were transmitted over thedata router 15 have been received by the leaves 21, thereby indicatingthat the data router 15 is empty. This can be used to indicate that amessage transfer operation has been completed, or that the router 15 isempty as a result of the assertion AFD(i,j) all-fall-down (i,j) signalso that a context switch can occur.

2. Control Network 14

As noted above, the control network 14 transfers program commands fromthe scalar processors 12 to the processing elements 11 and returnsstatus information to the scalar processors 12, and in addition performsselected types of arithmetic operations. The control network 14 will begenerally described in connection with block diagrams depicted in FIGS.4A and 4B, and with FIG. 5, which depicts the structure of a controlnetwork message packet.

With reference first to FIGS. 4A and 4B, the control network 14, likethe data router 15, is generally tree-structured, having a plurality ofcontrol network node groups 50(i,j) ("i" and "j" are integers) organizedin a plurality of levels each identified by the index "i" in referencenumeral 50(i,j). In the reference numeral 50(i,j), the index (j)distinguishes the diverse control network node group 50(i,j) at eachlevel "i." The tree structure of the control network 14 is generallysimilar to that of the data router 15. In particular, each controlnetwork node group 50(i,j) corresponds to a data router node group20(i,j) having the same values for indices "i" and "j", and connectionsamong control network node groups 50(i,j) follow the same pattern asconnections among data router node groups 20(i,j). Each control networknode group 50(1,j) in the lowest level is connected to one or moreleaves 21, in the same pattern as the connections in the data router 15.

Similar terminology will be used in describing the control network 14 aswas used in describing the data router 15 above. In particular, in thecontrol network 15 represented in FIG. 2A, the control network nodegroup 50(M,0) at the highest level "M" is termed the "physical root" ofthe tree. At each level "i", each control network node group 50(i,j) istermed the "parent" of control network node group 50(i-1,j) connectedthereto, and each control network node group 50(i-1,j) is termed a"child" of the control network node group 50(i,j) to which it isconnected. The control network node group 50(i,j) will also be a childof the control network node group 50(i+1,j) connected thereto. In oneparticular embodiment, each control network node group 50(i,j) in aparticular level "i" is connected to four child control network nodegroups 50(i-1,j), in which case the "fan-out" of the tree, that is, thenumber of children connected to each parent, is four. As indicated abovein connection with the data router 15, the fan-out need not be constant,but may vary from level to level and also among control network nodegroups 50(i,j) within the same level.

The structure of a control network node group 50(i,j), which is shown onFIG. 4B, differs from the structure of a data router node group 20(i,j).With reference to FIG. 4B, a control network node group 50(i,j) includesthree control network nodes 51(i,j,l), where "l" can have the values"P," "C₁ " or "C₂." Within a control network node group 50(i,j), thecontrol network nodes are connected so that control network node51(i,j,P) is parent of child control network nodes 51(i,j,C₁) and51(i,j,C₂). It will be appreciated that parent control network node51(i,j,P) of control network node group 50(i,j) is itself a child of acontrol network node 51(i+1,j,C₁) or control network node 51(i+1,j,C₂)of a control network node group 50(i,j) of the next higher level "i+1."Similarly, each child control network node 51(i,j,C) is a parent ofeither a leaf 21 or a control network node 51(i-1,j,P) of the next lowerlevel "i-1."

It should be noted that, in FIGS. 4A and 4B, the indices "j" for controlnetwork nodes 51(i,j,l) in each level increase from right to left. Inthe following, for each parent control network node 51(i+1,j,l), thechild control network node 51(i,j,l) connected thereto with the lowerindex "j" will be termed the "left" child, and the control network node51(i,j,l) with the higher index "j" will be termed the "right" child.

The control network node group 50(i,j) thus contains two sub-levels ofcontrol network nodes 51(i,j,l), one defined by parent control networknode 51(i,j,P), and the other defined by child control network nodes51(i,j,C₁) and 51(i,j,C₂). This enables the control network node groups50(i,j) to have the same connection pattern within the control network14 as the corresponding data router node groups 20(i,j) within the datarouter 15, while at the same time providing a two-child/one-parentconnection for the control network nodes 51(i,j,l) which simplifiesperformance of the arithmetic operations as described below.

As in the data router 15, each connection between control network nodes51(i,j,l) depicted in FIGS. 4A and 4B represents two unidirectional datapaths, which transfer message packets in opposite directions between therespective nodes.

As noted above, the scalar processors 12 use the control network 14 tobroadcast commands to the processing elements 11. In this operation, ascalar processor 12 transmits a message packet, which will be describedbelow in detail in connection with FIG. 5, to the control network node51(1,j,C) to which it is connected. The control network nodes transferthe message packet up the tree to the root, which then transmits themessage packet down the tree to its children. As each control networknode receives such a downwardly-going message packet, it transmits it toall of its children until the packet is delivered to the leaves 21. Thecontrol network 14 effectively broadcasts the message packet, and thusthe command, to all of the processing elements 11. It will beappreciated that the message packet will also be received at leaves 21comprising scalar processors 12 and input/output processors 13, butthese processors can be configured to ignore the packet.

As also noted above, the system 10 can be partitioned so as toeffectively constitute multiple independently-operable systems, eachincluding at least one scalar processor 12 and one or more processingelements 11. In partitioning the system 10, the scalar processor 12establishes a logical root in a control network node 51(i,j,l) in thecontrol network 14 which differs from the control network node 51(M,0,P)which constitutes the physical root. The logical root effectivelycomprises the root of a sub-tree whose leaves include the scalarprocessor 12 and one or more other leaves 21. If a control network node51(i,j,l) becomes a logical root, while it is a logical root its parentnode 51(i+1,j,l) in the control network 14 does not not transmitdownwardly-going message packets thereto.

Each control network node 51(i,j,l) includes a root flag 1407, which isdescribed in detail in connection with FIGS. 12A below. When the rootflag 1407 is set, the control network node 51(i,j,l) is a root of thecontrol network 15. If the control network node 51(i,j,l) is to be aphysical root, the root flag 1407 may be set by appropriate conditioningof an input signal that controls the control network node. To establisha control network node 51(i,j,l) as a logical root, the scalar processor12 transmits a control network message packet therefor up the treecomprising control network 14. The message packet includes a heightvalue identifying the level and sub-level at which the logical root isto be established. Each control network node 51(i,j,l) which receivesthe message packet determines whether the height value corresponds toits level and sub-level, and if not passes the message packet to thenext control network note 51(i,j,l) up the tree. When a control networknode 51(i,j,l) determines that the height value in the message packetcorresponds to its level and sub-level, it sets its root flag 1407 andbegins operating as a logical root as described above. In connectionwith that, the control network node 51(i,j,l) notifies its parentcontrol network node 51(i,j,l) that it is a logical root.

It will be appreciated that a control network node 51(i,j,l) operatingas a logical root of a partition may receive a message packet thatindicates that a control network node 51(i+x,j,m) at a higher level orsub-level is to operate as a logical root. A scalar processor 11 mayissue such a message to, for example, increase the number of processingelements 11 or scalar processors 12 in the partition. In that event, thecontrol network node 51(i,j,l) stops operating as a logical root.

To simplify the following description, the term "root node," which mayappear with or without the reference numeral 51(i,j,l), will be used tocollectively refer to the physical root control network node 51(M,0,P),in situations in with the control network 14 is not partitioned, and toa control network node 51(i,j,l) comprising a logical root in situationsin which the control network 14 is partitioned. If the control network14 is partitioned, the logical root node functions for the other controlnetwork nodes 51(i,j,l) in the partition substantially in the samemanner as the physical control network node 51(M,0,P) functions for thecontrol network nodes 51(i,j,l) in an unpartitioned control network 14.Otherwise stated, the physical root node can be considered as thelogical root node of a partition comprising the entire system 10.

As noted above, the control network 14 also performs several types ofarithmetic operations in response to control network message packetstherefor, including scan and reduce operations. Scan operations aregenerally described in Guy E. Blelloch, Scan Primitives and ParallelVector Models, (Ph.D. Dissertation, Massachusetts Institute ofTechnology: 1988). In a scan operation initiated by processing elements11 that are logically arranged in a particular ordering, such as withincreasing indices "i" in reference numeral 11(i) (with indicesincreasing, for example, from right to left as shown in FIG. 4B), thescan operation for a particular arithmetic operator "*" on items of data"D(i)" maintained by the processing element 11(i) produces at each ofthe successive processing elements 11 in the ordering the result "R(i)":

    R(i)=D(0) * D(1) * D(2) * . . . * D(i-1), with R(0)=0      [Eqn. 1]

In the scan operation, the arithmetic operator may constitute a numberof types of operators, including, for example, signed or unsignedaddition, OR, XOR (exclusive-OR) and MAX, the latter referencingdetermination of a maximum of a set of values.

To accommodate scan operations, each processing element 11 includes anup data processor 1421, a down data processor 1652, and a scan buffer1410, all of which will be described below in connection with FIGS. 12Athrough 12D-1. To initiate a scan operation, the processing elements 11transfer control network message packets therefor over the controlnetwork 14. The control network message packet provided by eachprocessing element 11(i) includes that processing element's data itemD(i).

With reference to FIG. 4B, each control network node 51(1,j,C₁) and51(1,j,C₂), on receiving a message packet from the processing elementsconnected thereto, loads the data from the left processing element, thatis, the processing element 11(i) with the index "i" being zero or aneven number, into its scan buffer 1410. In addition, the up dataprocessor 1421 of each control network node 51(1,j,C) performs thearithmetic operation on the data to generate a result that correspondsto the combination of the data received from the two processing elements11 connected thereto, combined according to the arithmetic operatorbeing used in the scan operation. The control network node 51(i,j,C)uses the value generated by the up data processor 1421 as data in amessage packet, which it transmits to its parent.

Each control network node 51(i,j,l), except for the root node, onreceiving message packets from both its left and right children,performs the same series of operations. In particular, each controlnetwork node 51(i,j,l) at each sub-level up to the root node:

(a) stores in its scan buffer 1410 the data in the control networkmessage packet that it receives from its left child control network node51(i-1,j,l); it will be appreciated that this value corresponds to thecombination of the data from the processing elements in the sub-tree ofthe control network 14 whose root is the left child control network node51(i-1,j,l), combined according to the arithmetic operator being used inthe scan operation, and

(b) performs, using its up data processor 1421 the operation, defined bythe arithmetic operator being used in the scan operation, in connectionwith data from both of its children to generate a value which ittransmits in a message to its parent. It will be appreciated that thisvalue corresponds to the combination of the data from the processingelements in both sub-trees of the control network 14 whose roots areboth child control network network nodes 51(i-1,j,l) connected thereto.

Thus, at the point at which all control network message packets havepropagated up the control network tree, the scan buffer 1410 at eachcontrol network node 51(i,j,l), other than the root node, contains avalue corresponding to the data provided by the processing elements 11in the sub-tree whose root is the node's left child, processed accordingto the scan operaiton's arithmetic operator.

The root node also receives message packets from both of its childrencontaining intermediate results for a scan operation, it transmitsmessage packets down the tree. The root node receives, from each child,a value corresponding to the data provided by the processing elements 11in the sub-tree whose root is the respective child, processed accordingto the scan operation's arithmetic operator. It will be appreciated thatthe value received from the left child control network node correspondsto the combination of the data from the processing elements in thesub-tree of the control network 14 whose root is that left child controlnetwork node, and the value received from the right control network nodecorresponds to the combination of the data from the processing elementsin the sub-tree whose root is the right control network node, in bothcases the data being combined according to the scan operation'sarithmetic operator.

To its left child, the root node transmits a message packet whose datahas the value zero. To its right child, the root node transmits a packetwhose data has the value received from the left child. As noted above,that value corresponds to the combination of the data from theprocessign elements in the sub-tree of the control network 14 whose rootis that left child control network node, combined according to the scanoperation's arithmetic operator.

When each control network node 51(i,j,l) below the root node receives acontrol network message packet from its parent, it

(a) uses the down data processor 1652 to generate a value correspondingto the value of the data received from the parent combined with theintermediate result stored in the nodes' scan buffer 1410 according tothe arithmetic operator used in the particular scan operation, which ittransmits in a control network message packet to its right child; itwill be appreciated that this value corresponds to the combination ofthe data from the processing elements 11 in all sub-trees of the controlnetwork 14 up to the one whose root is the left child of the controlnetwork node, combined according to the arithmetic operator being usedin the scan operation, and

(b) tranmsits a control network message packet to its left child whosedata has the same value as that received from the parent; it will beappreciated that this value corresponds to the combination of the datafrom the processing elements in all sub-trees of the control network 14up to the one whose root is the left child of the parent of the controlnetwork node, combined according to the arithmetic operator being usedin the scan operation.

Thus, the control network message packets transmitted by the controlnetwork nodes 51(i,j,l) down the tree will propagate the zero value downthe left side to the left-most processing element 11(0). The nextprocessing element 11(1) will receive the combination, as defined by thearithmetic operator, of the zero value propagated from the root node andthe value stored in the scan buffer 1410 of the control network node51(1,0,C₁), which corresponds to the value of the data transmitted bythe processing element 11(0).

The next processing element 11(2) will receive, as the left childconnected to the control network node 51(1,0,C₂) the value stored in thescan buffer 1410 of the control network node 51(1,0,P), which, as notedabove, corresponds to the combination, as defined by the scanoperation's arithmetic operator, of the data from the processingelements 11(0) and 11(1). The processing elment 11(3) will receive, asthe right child, the combination of that value and the value in the scanbuffer 1410 of control network node 51(1,0,C₂), which, as noted above,corresponds to the data provided by the processing element 11(2).Accordingly, the processing element 11(3) will receive the combination,as defined by the scan operation's arithmetic operator, of the data fromprocessing elements 11(0), 11(1) and 11(2).

It will be appreciated that the control network nodes 51 will combinethe data provided to the successive processing elements 11 in thesub-tree of the root node's left child similarly. Accordingly, eachprocessing element 11(i) in that sub-tree will receives a valuecorresponding to data from processing elements 11(i-1) through 11(0)combined according to the arithmetic operator of the particular scanoperation.

The control network nodes 51 in the sub-tree of the root node's rightchild also combine the data in the control network message packetprovided by their respective parents with the data in their respectivescan buffer 1410 in a similar manner. As noted above, the root nodetransmits to its right child a control network message packet includinga value corresponding to the combination of the data provided by theprocessing elements 11 in the sub-tree defined by the root node's leftchild, combined according to the scan operation's arithmetic operator.It will be appreciated that the control network message packetstransmitted by the control network nodes 51(i,j,l) in that sub-tree willpropagate that value down the left side of the sub-tree to the left-mostprocessing element 11(i), so that that processing element 11(i) alsoreceives a value corresponding to data from processing elements 11(i-1)through 11(0) combined according to the arithmetic operator of theparticular scan operation. Since the control network nodes 51(i,j,l) inthat sub-tree operate in a manner similar to those in the sub-treedefined by the root node's left child, each processing element 11(i)will receive a value corresponding to data from processing elements11(i-1) through 11(0) combined according to the arithmetic operator ofthe particular scan operation.

The control network 14 can also perform a backward scan operation, inwhich the scan direction is from right to left, that is, towardprocessing elements 11(i) of lower indices. In that case, eachprocessing element 11(i) will receive a value corresponding to data fromprocessing elements 11(i+1) through 11(N) (where "N" is the highestindex) combined according to the arithmetic operator of the particularscan operation. In that operation, each control network node 51(i,j,l)interchanges control network message packets that it receives at itsinput terminals from its children, and also the control network messagepacket that it transmits through the outputs to its children, andotherwise operates similar to that above. This effectively interchangesthe left and right children at each level, so that if the controlnetwork nodes 51 otherwise operate as described above, the scandirection will be reversed.

In addition, the control network 14 can perform a segmented scanoperation, in which the processing elements 11 of a partition may bedivided into two or more segments. In each case, the first processingelement 11(i) in the first segment is the first processing element 11(i)in the partition. The first processing element 11(i) in each succeedingsegment transmits a control network message packet in which a segmentbit is set. Each control network node 51(i,j,l) also includes a segmentflag 1561 (FIG. 12B-1G). Each control network node 51(i,j,l) operates asdescribed above, except that in transmitting control network messagepackets up the control network tree:

(a) if it receives a control network message packet from its right childin which the segment bit is set, it transmits in a control networkmessage packet to its parent data corresponding only to the data in thecontrol network message packet received from the right child; and

(b) if it receives a control network message packet from either child inwhich the segment bit is set, it sets its segment flag 1561, and setsthe segment bit in the control network message packet it that transmitsto its parent.

In either case, the control network node 51 buffers the data receivedfrom the left child control network node in its scan buffer 1410, in thesame manner as in an unsegmented scan operation as described above.

In connection with control network message packets that are transmitteddown the control network tree, each control network node 51, if itssegment flag 1561 is set, transmits to its right child a control networkmessage packet whose data corresponds to the value stored in the scanbuffer 1410. The control network node 51 transmits to it left child acontrol network message packet whose data corresponds to the data fromits parent, in the same manner as in an unsegmented scan operation asdescribed above.

It will be appreciated that the first processing element 11(i) which isthe first in each segment, other than the processing element 11(i)comprising the first in the partition, will not receive the value zero,as required in Eqn. 1 above. However, since those processing elements11, in initiating the scan operation, transmitted control networkmessage packets whose segment bits were set, they are aware that theyare the first processing elements 11(i) in their respective segments,and can interpret the value received as zero.

In a reduce operation for a particular arithmetic operator "*" on itemsof data "D(i)" maintained by the processing elements 11(i) produces atall of the processing elements 11 the same result "R":

    R=D(0) * D(1) * D(2) * . . . * D(i)                        [Eqn. 2]

In a reduce operation, the arithmetic operator may constitute a numberof types of operators, including, for example, signed or unsignedaddition, OR, XOR and determination of a maximum.

In performing a reduce operation, the processing elements 11 transfermessage packets therefor over the control network 14. The message packetprovided by each processing element 11(i) includes that processingelement's data item D(i). With reference to FIG. 4B, each controlnetwork node 51(1,j,C), on receiving a message packet from theprocessing elements connected thereto, performs the operation specifiedby the mathematical operator to generate an intermediate result, whichit transmits in a message packet to its parent node 51(1,j,P).

This operation is repeated at successive parent nodes at higher levelsin the tree comprising control network 14 until the message packetsreach the root node. When the root node receives message packets fromboth of its children, it performs the operation specified by themathematical operator on the data from its two children to generate aresult value. The root node generates message packets whose data is theresult value and transmits them to both of its children. Each of thecontrol network nodes 51(i,j,l) that receives such a message packetrepeats it to both of its children, until they reach the processingelements 11, thereby broadcasting the result to all of the processingelements 11.

As noted above, the leaves 21(i) may comprise a processing element11(i), a scalar processor 12(i) or an input/output processor 13(i). Inthe above description, only the processing elements 11(i) have beenindicated as engaging in scan operations and reduce operations. It willbe appreciated, however, that scalar processors 12(i) and input/outputprocessors 13(i) may, along with processing elements 11(i), engage insuch operations. Alternatively, the scalar processors 12(i) andinput/output processors 13(i) may abstain from the scan and reduceoperations. They may accomplish this either by transmitting controlnetwork message packets which contain data having a value of zero, or bytransmitting a special type of control network message packet, describedbelow as an abstain type, which the control network nodes 51(i,j,l) maytreat as containing data having the value zero.

As noted above, each processing element 11 maintains a message counterwhich counts data router message packets it transmits and receives overthe data router 15. The processing element 11 increments the messagecounter when it transmits a data router message packet over the datarouter 15 and decrements the counter when it receives a data routermessage packet over the data router 15 during a message transferoperation. It will be appreciated that during a message transferoperation some processing elements 11 may transmit more data routermessage packets than they receive, and thus at the end of the messagetransfer operation the message counter will have a positive value. Onthe other hand, some processing elements 11 may receive more data routermessage packets than they transmit during the message transferoperation, in which case the message counter will have a negative valueat the end of the messae transfer operation.

The processing elements 11 use the control network 14, in particularenabling a reduce operation, to determine when the data router 15 isempty, that is, when the data router 15 has delivered all data routermessage packets to processing elements 11. More specifically, eachprocessing element 11, after it transmits all of its data router messagepackets for the message transfer operation, begins transmitting controlnetwork message packets specifying a reduce operation, with signedaddition as the arithmetic operator. The data in each control networkmessage packet is the current value of the processing element's messagecounter. The processing elements 11 iteratively transmit such controlnetwork message packets until they receive a control network messagepacket whose data has the result value of zero. It will be appreciatedthat, at that point the processing elements 11 have collectivelyreceived as many data router message packets as they transmitted duringthe message transfer operation, and so the data router 15 will be emptyof data router message packets.

FIG. 5 depicts the structure of a control network message packet 60 thatis transferred over the control network 14. With reference to FIG. 5,the control network message packet 60 has a fixed length of thirteen"flicks." In one embodiment, each flick has five bits, with the firsttwelve flicks, identified as FLICK 0 through FLICK 11, including fourpacket information bits (labelled "PKT INFO" in FIG. 5) and one tag bit.The packet information portion of the first twelve flicks comprise apacket header portion 61 and a packet data portion 62. The thirteenthflick, namely FLICK 12 identified by reference numeral 63, contains achecksum used in error detection. The checksum is generated across allfive bits of the successive flicks in the packet 60. The tag bitscontain control information as described below.

The packet header portion 61 includes four fields, including a messagetype field 64, a packet type field 65, a combine function type field 66and a pattern field 67(0) and 67(1) (collectively identified byreference numeral 67). The packet data portion 62 includes eightfour-bit data nibbles 70(0) through 70(7) (generally identified byreference numeral 70) and a four-bit nibble 71 containing globalinformation.

The message type field 64 identifies the type of message contained inthe message packet 60. In one embodiment, a packet 60 can contain one offive different types of messages, including an SS (single source)message, an MS (multiple source) message, an ABS abstain message, anIDLE message and an NPAC nil packet message. When a scalar processor 12broadcasts a command to the processing elements 11 for processingthereby, it uses a single source message packet to carry the command. Inaddition, a scalar processor 12 may also use single source messagepackets to broadcast other types of control information to one or moreof the processing elements 11 or input/output processors 13, or toanother scalar processor 12.

A single source message packet is passed by each control network node51(i,j,l) which receives it up the control network tree from node tonode until it reaches the root node. The root node transmits the singlesource message packet down the tree to its children. Each controlnetwork node 51(i,j,l), which receives a single source message packetfrom its parent transmits it down the tree to both its children,effectively broadcasting the packet to all of the processing elements 11in the partition.

Multiple source messages are used by the processing elements 11 toinitiate scan and reduce operations as described above. Idle messagepackets are transmitted when a leaf 21 or control network node 51(i,j,l)has no other types of message packets to transmit. A leaf 21 transmitsabstain message packets to indicate that it is not participating in ascan or reduce operation. If a control network node 51(i,j,l) receivesidle or abstain message packets from both of its children, it maytransmit a message packet of the same type to its parent. If a controlnetwork node 51(i,j,l) receives a multiple source message packet fromone of its children and an abstain message packet from its other child,it does not thereafter wait for a multiple source message packettherefrom to use in the arithmetic operation specified in the multiplesource message packet that it receives from the one child. Instead, thecontrol network node 51(i,j,l) forwards the multiple source messagepacket that it receives to its parent, and, if the abstain messagepacket came from its right child, stores the data from the messagepacket in its scan buffer 1410.

A message packet of the nil packet type, unlike message packets of othermessage types, is only one flick in length. In particular, a nil packetmessage comprises only the message type flick 64, the contentsindicating that the message packet is of the nil packet type. A controlnetwork node 51(i,j,l) continually transmits messages of the nil packettype to its parent while it [that is, the control network node51(i,j,l)] is a logical root of a partition, and the parent transmitsmessage packets of the same type to that child. If the parent receives amultiple source message packet from its other child, it forwards it toits parent.

The packet type field 65, combine function type field 66 and a patternfield 67 contain further information about the information in thecontrol network message packet 60.

In one particular embodiment, the processing elements 11 can operate intwo operational modes, identified herein as "supervisor" and "user." Ifthe message type field 64 indicates that the control network messagepacket is a single source message packet, the packet type field 65 canidentify a message packet as a broadcast supervisor packet or abroadcast user packet. If the packet type field 65 indicates that thecontrol network message packet is a broadcast supervisor packet, itcontains a command for execution by the processing elements 11 in thesupervisor mode. On the other hand, if the packet type field indicatesthat the control network message packet contains a broadcast userpacket, it contains a command for execution by the processing elements11 in the user mode.

In addition, if the message type field 64 indicates that the controlnetwork message packet is a single source message packet, the packettype field 65 may indicate that the control network message packet is aninterrupt packet. The interrupt packet may be used to initiateoperations at particular ones of the processing elements 11. Theoperations and the particular ones of the processing elements 11 toperform them may be identified in the packet data portion 62.

Further, if the message type field 64 indicates that the control networkmessage packet is a single source message packet, the packet type field65 may indicate that the control network message packet containsconfiguration information which enables the establishment or eliminationof a logical root at a particular control network node 51(i,j,l). If thepacket type field identifies the message packet as containingconfiguration information, the first two flicks 70(0) and 70(1) ofpacket data portion 62 contain data specifying the level and sub-levelin control network 14 at which the logical root is to be established.The control network node 51(i,j,l) at that level and sub-level whichreceives the configuration message packet establishes itself as thelogical root.

If the message type field 64 identifies the messge packet as a multiplesource message packet, the packet type field 65 identifies the operationto be performed as a scan involving data in a single packet or aplurality of packets, or to perform an operation to determine whetherthe data router 15 is empty. The data to be used is contained in datafields 70(0) through 70(7) (generally identified by reference numeral70) of the packet data portion 62. If the packet type field 65identifies a scan operation involving data in a single packet, the scanoperation is limited to a data value having a single thirty-two bitword. However, if the packet type field identifies a scan operationinvolving data in a plurality of successively-transmitted packet, whichwill be identified as a "multi-word scan," the scan operation involvesdata values of more than thirty-two bits, which are contained in controlnetwork message packets 60 successively transmitted by the processingelements 11. In either case, if the packet type field 65 identifies theoperation as a scan operation, the pattern field 67 further identifiesit as either a scan forward or scan backward operation or a reduceoperation, and combine function type field 66 identifies the particulararithmetic operator to be used in the operation.

As has been described above, control network message packets of themultiple source type may be used, with arithmetic operations, todetermine whether the data router 15 is empty, using the contents ofmessage counters maintained by the processing elements 11 as data.Similar control network message packets may also be used to performother control operations using, for example, bits of the globalinformation field 71. For example, the scalar processors 12 may need tobe notified when all of the processing elements 11 have finishedexecuting a particular command before they transmit a subsequentcommand. In that case, each processing element when it has finishedexecuting a command, may tranmsit a control network message packet 60,of the multiple source type, indicating a reduce operation using the ORoperator, with a particular bit in the global information field 71 beingset. It will be appreciated that, after all of the processing elements11 have executed the istruction and transmitted corresponding packets,the root node will as the result of the reduce operation, broadcastcontrol network message packets down the control network tree in whichthe bit will be set. When the scalar processor 12 receives the resultingcontrol network message packet from the control network node 51(1,j,l)connected thereto, it can determine the condition of the bit anddetermine therefrom that the command has been executed.

Bits of the global information field 71 may also be used by theprocessing elements 11. In processing certain commands from the scalarprocessors 12, the processing elements 11 sometimes may reach a point inprocessing a command at which they have to verify that all of theprocessing elements have reached the same point before they proceed. Toaccomplish that, when each processing element has reached the particularprocessing point it may transmit a control network message packet asdescribed above, that is, of the multiple source type, indicating areduce operation using the OR operator, with a particular bit in theglobal information field 71 being set. When the processing elements 11receive the resulting control network message packet from theirrespective control network nodes 51(1,j,l) connected thereto, they candetermine therefrom that all of the processing elements 11 have reachedthe required point in their processing of the command, and continueprocessing.

The tag bits of the successive flicks in a control network messagepacket 60 contain various types of control and status information.Several of the tag bits control the flow of control network messagepackets through the control network 14. Five tag bits comprise scan flowbits, generally identified by reference numerals 72(i) ("i" is aninteger from "1" through "5"). The control network nodes 51(i,j,l),processing elements 11 and scalar processors 12, as wll as anyinput/output processors 13 which transmit and receive control networkmessage packets over the control network 14, use the scan flow bits tocontrol the transfer of message packets between directly-connectedcomponents in the control network 14.

Two tag bits, including a broadcast user flow bit 73 and a broadcastsupervisor flow bit 74 are conditioned by the processing elements 11,scalar processors 12 and those input/output processors 13 which transmitcontrol network message packets over the control network 14, to indicatewhether they are able to receive control network message packetscontaining control information for the supervisor and user modesrespectively. Each processing element 11, scalar processor 12 andinput/output processor 13, respectively, conditions bits 73 and 74 inany control network message packets that it transmits to indicatewhether it can receive single source message packets having packettypes, as indicated in packet type field 65, of broadcast supervisortype and broadcast user type, respectively.

Another tag bit that controls the control network 14 is a flush bit 75.When a control network node 51(i,j,l) receives a control network messagepacket in which the flush bit 75 is set, it clears its scan buffer. Thismay be used to clear intermediate results of a scan or reduce operationfrom the control network 14 during a context switch.

A soft error bit 76 is used by a control network node 51(i,j,l) toindicate that it has detected a software error from the contents of acontrol network message packet 60. For example, if the control networknode 51(i,j,l) determines that the contents of the packet type field 65do not identify one of the established packet types for the message typeidentified in message type field 65, the node may set the soft error bit76.

As described above, the control network 14 performs segmented scanoperations using data in message packets transmitted by the processingelements 11. A segment bit 77, when set, indicates that the controlnetwork message packet 60 contains data for the upper end of a segment.A scan overflow bit 80, when set, indicates that the result of thearithmetic operation is larger than can be accommodated in the datafields 70 of the control network message packet 60. The scan overflowbit 80 may also be used to indicate overflow during a reduce operation.If the scan overflow bit 80 is set, the operation can be repeated in amulti-word operation.

Finally, a control network message packet 60 includes an AFDall-fall-down bit 81. If a control network node 51(i,j,l) receives acontrol network message packet 60 in which the AFD all-fall-down bit 81is set, it asserts an ADF(i,j,l) all-fall-down signal. The AFD(i,j,l)all-fall-down (i,j) signal from each parent control network node51(i,j,P) is connected to the data router nodes 22(i,j,k) of the datarouter node group 20(i,j) having the same indices "i" and "j."

3. Diagnostic Network 16

As noted above, the diagnostic network 16, under control of a diagnosticprocessor, facilitates testing of other portions of the system 10 toidentify, locate and diagnose defects. In addition, the diagnosticnetwork 16 may be used to establish selected operating conditions in theother portions of the system 10 as described below. The generalstructure of the diagnostic network 16, and its connection to the otherelements of the system 10, will be described in connection with FIGS. 6Athrough 6C. Messages transferred over the diagnostic network 16 will bedescribed in connection with FIG. 7.

With reference to FIGS. 6A through 6C, the diagnostic network 16includes a plurality of diagnostic network node generally identified byreference numeral 100(h,p,r-l), where "h" and "p" comprise integersrepresenting a height value and a pod-type value, and "r-l" comprisesone or more integers which together comprise a root-leaf value. Thevarious diagnostic network nodes 100(h,p,r-l) are connected in atree-type structure which actually forms a tree of trees as shown in theFigs. In particular, the diagnostic network 16 includes a high-ordertree identified as a height-decoding tree, as represented by thediagnostic network nodes 100(h,p,r-l) in the left-most columns of therespective FIGS. 6A through 6C. Each diagnostic network node100(h,p,r-l) in the height decoding tree is identified by a referencenumeral 100(h,0,0 . . . 0), where the value of "h" is associated with alevel in the data router 15 and control network 14. A diagnosticprocessor 101 is connected to the diagnostic network node 100(h,0,0 . .. 0) at the highest level of the height decoding tree.

The height decoding tree is essentially a linear tree, that is, there isno fan-out from level to level in the height decoding tree. The heightdecoding tree essentially forms the backbone of other lower-level treesin the diagnostic network 16, including a pod-type decoding tree,represented by diagnostic network nodes 100(h,p,r-l) in the middlecolumn of FIGS. 6A through 6C, and a root-leaf decoding tree representedby diagnostic network node 100(h,p,r-l) in the right-hand column of FIG.6A through 6C. In particular, depending from each diagnostic networknode 100(h,0,0 . . . 0) in the height decoding tree is a diagnosticnetwork node 100(h,1,0 . . . 0), which comprises the pod-type decodingtree. Although only one diagnostic network node 100(h,1,0 . . . 0) isshown in the pod-type decoding tree at each level, the diagnosticnetwork 16 may include multiple decoding nodes connected in a treestructure. In that case, the diagnostic network node 100(h,1,0 . . . 0)will comprise the root of the pod-type decoding tree, and otherdiagnostic network nodes 100(h,p,0 . . . 0) will comprise intermediatenodes and leaves of the pod-type decoding tree.

In addition, depending from diagnostic network nodes 100(h,1,0 . . . 0)in the pod-type decoding tree are diagnostic network nodes 100(h,p,r-l)comprising the root-leaf decoding tree. As shown in FIGS. 6A through 6C,depending from each diagnostic network node 100(h,1,0 . . . 0) in thepod-type decoding tree is a one or more trees of diagnostic networknodes 100(h,p,r-l) in the root-leaf decoding tree. In the embodimentdepicted in FIGS. 6A through 6C, each diagnostic network node100(h,p,r-l) can accommodate a fan-out of two, and so if the pod-typedecoding tree includes one diagnostic network node 100(h,1,0 . . . 0),the diagnostic network 16 at that level may include up to two root-leafdecoding trees, which may connect to diverse types of other componentsin the system 10. Each root-leaf decoding tree includes a rootdiagnostic network node 100(h,p,r . . . 0) connected to the pod-typedecoding tree, and extends to a plurality of leaf diagnostic networknodes 100(h,p,r-l) connected to a particular type of pods in the system10.

The portions of system 10 comprising "pods" may depend upon the physicalembodiment of the particular system. As depicted on FIGS. 6A through 6C,the data router nodes 22(i,j,k) may comprise one type of pod, thecontrol network nodes 51(i,j,l) may comprise a second type of pod, andthe leaves 21 may comprise a third type of pod. As shown in FIG. 6A,level "M," which corresponds to the root level of the control network 14and data router 15, includes two root-leaf decoding trees. One root-leafdecoding tree comprises the diagnostic network nodes identified byreference numerals 100(M,1,1 . . . 0) through 100(M,1,r-l), which isconnected to the pods of the data router nodes in the root data routernode group 20(M,0). The other root-leaf decoding tree comprises thediagnostic network node identified by reference numeral 100(M,2,1 . . .0), which is connected to the pod comprising the root control networknode group 50(M,0).

Similarly, level "M-1," which corresponds to one level below the rootlevel of the control network 14 and data router 15, also includes tworoot-leaf decoding trees. One root-leaf decoding tree comprises thediagnostic network nodes identified by reference numerals 100(M-1,1,1 .. . 0) through 100(M,1,r-l), which is connected to the pods of the datarouter nodes in the data router node groups 20(M-1,j), one level belowthe root level. The other root-leaf decoding tree comprises thediagnostic network nodes identified by reference numerals 100(M,2,1 0 .. . 0), 100(M,2,1 1 . . . 0), and 100(M,2,1 2 . . . 0) which-areconnected to the pods comprising the root control network node group50(M,0). The other levels of the diagnostic network 16, down to level"1," which corresponds to the lowest levels in the control network 14and data router 15, are similar, including two root-leaf decoding trees,one connected to pods comprising the data router node groups 20(i,j) andthe other connected to pods comprising the control network node groups50(i,j).

As indicated above, the diagnostic network 16 also includes a level "0"connected to leaves 21 in the system 10. That level includes only oneroot-leaf decoding tree, comprising the diagnostic network nodes100(0,1,1 . . . 0) through 100(0,1,r-l), all of which are connected toleaves 21.

A "pod" may comprise an individual data router node 22(i,j,k), controlnetwork node 50(i,j,l) or leaf 21, or groups thereof. In one particularembodiment, a "pod" is a "field-replaceable unit," such as an entirecircuit board, which is replaceable by field-service or maintenancepersonnel. In that embodiment, the diagnostic network 16 can diagnoseand locate failures in such field-replaceable units.

It will be appreciated that, if a pod-type decoding tree at anyparticular level includes multiple diagnostic network nodes 100(h,p,0 .. . 0) organized in a tree structure, multiple the root-leaf decodingtrees can be provided each depending from a node comprising a leaf ofthe pod-type decoding tree. Thus, for example, if a particular level inthe diagnostic network 16 required three or four root-leaf decodingtrees, each connected to pods of particular types, if the fan-out fromeach level to the next in the pod-type decoding tree is two, thepod-type decoding tree would include at least three diagnostic networknodes 100(h,p,r-l), including a root node and two leaf nodes connectedthereto. In that case, each leaf node would be able to connect to tworoot-leaf decoding trees. It will be appreciated that, if the fan-outsin each of the trees is different from two, the number of levels andnumber of nodes in each level within each tree may also differ from thatspecifically described herein. In one particular embodiment, fan-outs inparticular diagnostic network nodes 100(h,p,r-l) of both two and eightare used, at different levels in the respective trees comprising thediagnostic network 16.

The diagnostic network nodes 100(h,p,r-l) are generally similar, andwill be described in detail in connection with FIGS. 13A through 13C. Inbrief, each diagnostic network node 100(h,p,r-l) includes an addresscontrol portion, generally identified by reference numeral 102, and adata control portion, generally identified by reference numeral 103. Theaddress control portion of diagnostic network node 100(M,0,0 . . . 0)receives address control signals from the diagnostic processor over abus 104(P). The node uses the address control signals to establishaddress state in an address state store 105.

The address state maintained by the diagnostic network node 100(M,0,0 .. . 0) enables it to transmit subsequently-received address controlsignals (a) to one child node, in this case node 100(M-1,0,0 . . . 0)over a bus 104(C₁), (b) to the other child node, in this case node100(M,1,0 . . . 0) over a bus 104(C₂, (c) to both child nodes over thesame buses, or, alternatively, (d) to neither child node. The node'saddress control portion 102 includes flags 106(C₁) and 106(C₂) eachassociated with a corresponding bus 104(C₁) and 104(C₂). If the flag106(C_(i)) is set in response to the received address control signals,the node is enabled to thereafter transmit the address control signalsto the respective child node over a bus 104(C_(i)), and otherwise it isclear.

The diagnostic processor 101 controls the conditioning of each of theflags 106(C_(i)) in the state store 105 of diagnostic network node100(M,0,0 . . . 0) serially. After the address state has beenestablished in the state store 105 of diagnostic network node 100(M,0,0. . . 0), the node transmits the address control signals that itsubsequently receives over bus 104(P) from the diagnostic processor 101over the particular buses 104(C_(i)) whose flags 106(C_(i)) are set. Ifboth flags 106(C_(i)) are set, the diagnostic network node 100(M,0,0 . .. 0) transmits the address control signals over both buses 104(C_(i)) inparallel. The address control signals thereafter enable either or bothof those nodes to condition the flags 106(C_(i)) in their respectiveaddress state stores 105, enabling them to thereafter transmit theaddress control signals received thereby to either or both of thediagnostic network nodes 100(h,p,r-l) connected thereto. This processcontinues until flags 106(C_(i)) are set in selected ones of the leafdiagnostic network nodes 100(h,p,r-l) in the root-leaf decoding tree.This process may be repeated any number of times to condition flags106(C_(i)) in any combination of the leaf diagnostic network nodes100(h,p,r-l).

The sequence of flags 106(C_(i)) that are set in the various diagnosticnetwork nodes 100(h,p,r-l), from the root diagnostic network node100(1,0,0 . . . 0) in the height decoding tree to the leaf diagnosticnetwork nodes 100(h,p,r-l) in the root-leaf decoding trees, essentiallyform paths from the diagnostic processor 101 to selected pods. The pathsmay be subsequently used to carry diagnostic test data in parallel fromthe diagnostic processor to the selected pods, and to return testresults.

After it has conditioned flags 106(C_(i)) in the various diagnosticnetwork nodes 100(h,p,r-l), the diagnostic processor 101 may alsoretrieve the state from each of the diagnostic network nodes100(h,p,r-l). After each flag 106(C_(i)) is conditioned, the diagnosticnetwork node 100(h,p,r-l) may transmit a signal representing its stateits state over its bus 104(P), which is coupled up the tree to thediagnostic processor 101. If multiple flags are conditioned in diversenodes in parallel, the diagnostic processor 101 transmits an expectedaddress data signal, which enable the nodes intermediate the originatingnodes and the diagnostic processor to combine the signals representingthe state of the respective flags in response to a control signal fromthe diagnostic processor 101.

Thus, if the flags 106(C_(i)) whose conditions are being retrieved areto be set, resulting in asserted state signals, the diagnostic processor101 may enable the intermediate nodes to logically AND the flag statesignals received from their child nodes. In that case, if anintermediate node receives a negated state signal, indicating that theflag 106(C_(i)) whose condition is received is, erroneously, not set,the node will provide a negated state signal, which will be propagatedup the tree to the diagnostic processor 101. On the other hand, if theflags whose conditions are being retrieved are to be cleared, resultingin negated state signals, the diagnostic processor 101 may enable theintermediate nodes to logically OR the flag state signals received fromtheir child nodes. In that case, if an intermediate node receives anasserted state signal, indicating that the flag 106(C_(i)) whosecondition is received is, erroneously, not clear, the node will providean asserted state signal, which will be propagated up the tree to thediagnostic processor 101.

After the diagnostic processor 101 has established the address states inthe respective diagnostic network nodes 100(h,p,r-l) to selected pods,it may transmit a test data out signal and an expected test data controlsignal, which are received by the root diagnostic network node 100(M,0,0. . . 0), over a bus 110(P). The root diagnostic network node 100(M,0,0. . . 0) transmits the received signals over respective buses 110(C₁)and 110(C₂), as determined by the states of the respective flags106(C_(i)), and the other diagnostic network nodes do the same. Thus,the diagnostic network nodes 100(h,p,r-l) couple the test data outsignal and expected test data control signal down the respective treesalong paths defined by the set flags 106(C_(i)). At some point, at leastsome of the leaf diagnostic network nodes 100(h,p,r-l) will couple testdata signals to the selected pods, and obtain test data out signalsrepresenting diagnostic test results.

The diagnostic network nodes 100(h,p,r-l) will pass the test data outsignals up the paths defined by the set flags 106(C_(i)), each nodecombining the test data out signals received from its children inresponse to the expected test data control signal in a manner similar tothat described above in connection with retrieval of the states of therespective flags. That is, if the test data out signal is expected to beasserted, the diagnostic processor 101 may enable the nodes to logicallyAND the test data signals received from the pods or child nodesconnected thereto. In that case, if an intermediate node receives anerroneous negated test data out signal, the node will provide a negatedtest data out signal to its parent, which will be propagated up the treedefining the diagnostic network 16 to the diagnostic processor 101. Onthe other hand, if the test data out signal is expected to be negated,the diagnostic processor 101 may enable the intermediate nodes tologically OR the test data out signals received from the pods or thechild nodes connected thereto. In that case, if an intermediate nodereceives an erroneous asserted test data out signal, the node willprovide an asserted test data out signal to its parent, which will bepropagated up the tree to the diagnostic processor 101.

If the diagnostic processor 101 receives an erroneous test data outsignal, it can thereafter repeat the operations in connection withsubsets of the previously-selected pods to identify the one whichprovided the erroneous signal. In that operation, the diagnosticprocessor 101 establishes states of the address flags 106(C_(i)) in thediagnostic network nodes 100(h,p,r-l) to establish paths therethrough toa selected subset and repeats the test operation in connection with thatsubset. If the test data out signal indicates an erroneous result, thediagnostic processor 101 can reduce the size of the subset and repeatthe operation. If the test data out signal indicates a correct result,on the other hand, the diagnostic processor 101 can repeat the operationin connection with a different subset. In one embodiment, the diagnosticprocessor 101 performs a binary search operation, iteratively repeatingthe operation in connection with half of the pods selected during theprevious iteration to locate the pod providing the erroneous test dataout signal.

Although not shown in FIGS. 6A through 6C, the diagnostic network 16 mayinclude multiple diagnostic processors connected to various ones of thediagnostic network nodes 100(h,p,r-l). Each diagnostic processor mayselectively control the portions of the tree defining the diagnosticnetwork 16 below the diagnostic network node 100(h,p,r-l) connectedthereto. Alternatively, the diagnostic processors may selectivelycondition the diagnostic network nodes 100(h,p,r-l) connected thereto toreceive signals from, and transmit signals to, their respective parentdiagnostic network nodes 100(h,p,r-l). The additional diagnosticprocessors may facilitate diverse diagnostic operations in various partsof the system 10 in concurrently.

In one specific embodiment, the interface between the leaf diagnosticnetwork nodes 100(h,p,r-l) and the pods comprises the interface definedby the Joint Test Action Group ("JTAG"), as described in IEEE Std.1149.1 (hereinafter "JTAG specification"). In any event, the interfaceprovides a serial scan chain circuit in each pod. The serial scan chaincircuit in each pod may extend through a number of registers and otherstorage elements in the respective pods, and may be used to establishthe states thereof to thereby establish selected operating conditions inthe respective pods. For example, each data router nodes 22(i,j,k) andcontrol network nodes 51(i,j,l) uses height signals identifying therespective levels, which may be provided by a register thereon that canbe loaded through the serial scan chain circuit. These nodes also usesignals which indicate whether connections to the respective parent orchild nodes are enabled or disabled, which may also be provided byregisters loaded through the serial scan chain circuit.

FIG. 7 depicts the structure of a diagnostic message packet 120.Diagnostic message packets 120 differ from the data router messagepackets 30 and control network message packets 60, in that they are notgenerated by pods connected to the diagnostic network 16 for deliverythrough the diagnostic network 16 to other pods connected thereto. Thediagnostic network message packets are generated by a diagnosticprocessor for delivery to the pods, which, in turn, generate responsedata for transmission to the diagnostic processor.

In any event, the diagnostic message packet 120 includes an addressportion 121 and a test data portion 122. The address portion 121conditions the respective address control portions 102 in the diagnosticnetwork nodes 100(h,p,r-l). The test data portion 122 is represented bythe test data in and and test data out signals. along with the expectedtest data in signals, coupled through the data control portions 103 ofthe respective diagnostic network nodes 100(h,p,r-l). Depending on thelocation of the diagnostic processor generating the diagnostic messagepacket 120, the packet 120 may include three sections in the addressportion 121, including a height identification portion 123, a pod-typeidentification portion 124 and a root-leaf identification portion 125.Each of the portions 123 through 125 are used by diagnostic networknodes 100(h,p,r-l) in the respective height, pod-type and rootleafdecoding trees to condition the respective flags 106(C_(i)) therein. Itwill be appreciated that the length of the respective portions 123through 125 will vary, depending upon the number of diagnostic networknodes 100(h,p,r-l) whose flags 106(C_(i)) are to be conditioned, and thenumber of flags in each node.

II. Detailed Description of Particular Circuits

A. General

The remainder of this specification will present details of circuitsused in one embodiment to carry out the invention as set forth in theclaims. In the following, the detailed logic of a leaf 21, in particulardetails of connection of a leaf 21 to the control network 14 and datarouter 15 will be discussed in connection with FIGS. 8 through 10G.Thereafter, the detailed logic of a data router node 22(i,j,k) will bedescribed in connection with FIGS. 11A through 11D, the detailed logicof a control network node 51(i,j,l) will be described in connection withFIGS. 12A through 12D-1, and the detailed logic of a diagnostic networknode 100 will be described in connection with FIGS. 13A through 13C.

B. Leaf 21

1. General

FIG. 8 is a general block diagram of a leaf 21, in particular, aprocessing element 11 in the computer system 10 depicted in FIG. 1.Other types of leaves, including a scalar processor 12 and aninput/output processor 13 are generally similar at a block diagramlevel, except as noted below.

With reference to FIG. 8, processing element 11 includes a processor200, memory 201 and network interface 202 all interconnected by a memorybus 203. The network interface 202 interfaces the processing element 11to the various communication mechanisms 14, 15 and 16 in system 10. Inparticular, the network interface 202 includes a control networkinterface 204 (described below in more detail in connection with FIGS.10A-1 through 10G) that receives (ejects) control network messagepackets 60 from the control network 14, and that transmits (injects)control network message packets 60 to the control network 14. Similarly,a data router interface 205 (described below in more detail inconnection with FIGS. 9A-1 through 9D-7) receives (ejects) data routermessage packets 30 from the data router 15 and transmits (injects) datarouter message packets 30 to the data router 15, and a diagnosticnetwork interface 206 receives diagnostic network message packets fromthe diagnostic network 16 and transmits diagnostic network results overthe diagnostic network 16. FIG. 14 depicts the logic diagram of aninterface circuit that may be used as the diagnostic network interface206 to interface the network interface 202 to the diagnostic network 16.

As noted above, scalar processors 12 and input/output processors 13 aregenerally similar, at a block diagram level, to the processing element11 depicted on FIG. 8. Scalar processors 12 may also include, forexample, video display terminals (not shown) which may comprise consolesto allow control of the system 10 by an operator. In addition, scalarprocessors 12 may include such elements as, for example, magnetic diskstorage subsystems (also not shown) to store programs and data to beprocessed by the processor. It will be appreciated that processingelement 11 may also include such elements. As noted above, aninput/output processor 13 will include interfaces to external datainput/output and storage devices, including frame buffers, magnetic diskstorage devices, and other such elements that are well known in the art.

The network interface 202 includes a clock buffer 207 that receives theSYS CLK system clock signal from the clock circuit 17 and generates aNODE CLK node clock signal in response. In one particular embodiment,the clock buffer 207 comprises a buffer as described in U.S. patentappn. Ser. No. 07/489,079, filed Mar. 5, 1990, in the name of W. DanielHillis, et al., entitled Digital Clock Buffer Circuit ProvidingControllable Delay, and assigned to the assignee of the presentapplication. The network interface 202 uses the NODE CLK node clocksignal to synchronize its operation with the control network 14, datarouter 15, and diagnostic network 16. The NODE CLK node clock signal mayalso be used in generating clock signals for controlling the othercomponents of the processing element 11 shown in FIG. 8, but it will beappreciated that those components may alternatively be controlled bysignals other than the NODE CLK node clock signal.

The memory bus 203 transfers address signals that define a processingelement address space. The memory 201 includes a memory controller 208and a plurality of memory banks generally identified by referencenumeral 210, the memory banks 210 including a plurality of addressablestorage locations within the processing element address space. Inaddition, the control network interface 204 and data router interface205 include a plurality of registers, described in more detail below,which are also within the processing element address space.

The interfaces 204, 205 and 206 are connected through a bus 211 to aprocessing element interface 212, which, in turn, is connected to thememory bus 203. In response to receipt of control network messagepackets 60 from the control network 14 or diagnostic network messagepackets 30 from the data router 15, the processing element interface 212can interrupt the processor 200. In response to the interrupt, theprocessor 200 can, by reading appropriate registers in the respectiveinterface 204 or 205, retrieve the contents of the packet from thenetwork interface 202. The processor may store the retrieved packetcontents in the memory 201.

In addition, the processor 200 can initiate transfer of a controlnetwork message packet 60 over the control network 14 or a data routermessage packet 30 over the data router 15. In this operation, theprocessor 200 transmits packet information over bus 203 to particularregisters in the network interface 202. The processing element interface212, in response to address signals over memory bus 203 identifying theregisters, receives the packet information and loads it into therespective registers. Upon receiving the packet information, therespective interface 204 or 205 initiates transmission of a messagepacket 60 or 30 over the respective control network 14 or data router15.

The processor 200 executes the commands transmitted in control networkmessage packets 16 over the control network 14 by the scalar processors12 and received by the control network interface 204. In response to acommand, the processor 200 processes one or more instructions, which aremaintained in memory 201, which may enable the processor 200 to processdata in the memory 201. In addition, the instructions may enable theprocessor 200 to transmit packet information to respective registers inthe network interface 202 to initiate a transfer of a packet 30 or 60over the respective data router 15 or control network 14, or to readinformation from respective registers to thereby retrieve the receivedpacket information.

2. Data Router Interface 205

i. General

The details of data router interface 205 will be described in connectionwith FIGS. 9A-1 through 9D-7. With reference to FIG. 9A-1, the datarouter interface 205 includes a data router message injector portion220, a message ejector portion 221 and an injector/ejector commoncontrol/status portion 222, all connected to processing elementinterface bus 211. The data router message injector portion 220 injectsdata router message packets 30 over the data router 15; that is, ittransmits data router message packets 30 to the data router nodes22(1,j,0) and 22(1,j,l) connected thereto.

The data router message injector portion 220 includes two messageinjector ports identified as left message injector port 223(l) and rightmessage injector port 223(r) for injecting message packets 30 into thedata router 15. In the following, data router node 22(1,j,0) is termedthe "left" node, and data router node 22(1,j,l) is termed the "right"node; in that case, left message injector port 223(l) is connected totransmit data router message packets to data router node 22(1,j,0) andright message injector port 223(r) is connected to transmit data routermessage packets to data router node 22(1,j,1).

Data router message injector portion 220 also includes an injectorcommon control/status portion 224 that connects to, and controls certainoperations of, both left and right message injector ports 223(l) and223(r). For example, when the processor 200 initiates transmission of adata router message packet 30, it may specify that the message packet 30be transmitted through either the left or the right message injectorport 223(l) or 223(r). In that case, the data router interface 205 willtransmit the packet 30 through the specified port 223(l) or 223(r).Alternatively, the processor may not specify the particular port 223(l)or 223(r), in which case the injector common control/status portion 224will select one of the ports 223(l) or 223(r) to transmit the packet 30.

The message ejector portion 221 receives and buffers data router messagepackets 30 from the data router 15. In addition, the message ejectorportion 221 may initiate interrupting of the processor 200 on receivinga new data router message packet, and it transmits the buffered packetsover the processing element interface bus 211 in response to a retrievalrequest from the processor 200. The message ejector portion 221 includesa left message ejector port 225(l) and a right message ejector port225(r) that are connected to receive data router message packets 30 fromdata router nodes 22(1,j,0) and 22(1,j,1), respectively.

Data router message ejector portion 221 also includes an ejector commoncontrol/status portion 226 that connects to, and controls certainoperations of, both left and right message ejector ports 225(l) and225(r). For example, if both right and left ejector ports 225(l) and225(r) have received message packets 30 and the processor 200 hasrequested that the message data be transmitted to it without identifyingeither the particular left or right ejector port 225(l) or 225(r), theejector common control/status portion 226 determines the order in whichthe ports 225(l) and 225(r) will transmit the packets over theprocessing element interface bus 211.

To transmit a data router message packet 30 to the data router node22(1,j,0) connected thereto, the left message injector port 223(l), insynchrony with the NODE CLK node clock signal, iteratively transmits (L)IN FLIT left inject flit signals to transmit successive flits of thepacket 30 to the data router node 22(1,j,0). The left message injectorport 223(l) may transmit while the data router node 22(1,j,0) isasserting an (L) IN FLY left input fly signal; if the data router node22(1,j,0) negates the (L) IN FLY left input fly signal the left messageinjector port 223(l) stops transmitting. The right message injector port223(r) transmits similar (R) IN FLIT right inject flit signals to datarouter node 22(1,j,1) in response to an asserted (R) IN FLY right inputfly signal.

The left message ejector port 225(l), in synchrony with the NODE CLKnode clock signal, iteratively receives (L) OUT FLIT left eject flitsignals to for successive flits of the packet 30 from the data routernode 22(1,j,0). The left message ejector port 225(l) may enable the datarouter node 22(1,j,0) to transmit by asserting an (L) OUT FLY left ejectfly signal; if the port 225(l) negates the (L) OUT FLY left eject flysignal the data router node 22(1,j,0) stops transmitting. The datarouter node 22(1,j,1) transmits similar (R) OUT FLIT right eject flitsignals to right message ejector port 225(r) in response to an asserted(R) OUT FLY right eject fly signal.

FIGS. 9A-2A and 9A-2B depict the registers in the control/statusportions 222, 224 and 226 in the data router interface 205. FIG. 9A-2Adepicts the details of a data router interface middle register set 230which is used by the processor 200 when it does not specify theparticular message injector port 223(l) or 223(r) to transmit aparticular data router message packet 30, or the message ejector port225(l) or 225(r) from which it is to receive a data router messagepacket 30. With reference to FIG. 9A-2A, register set 230 includes twostatus and control registers, including a status register 231 and aprivate register 232, a receive register 233, and two transmitregisters, namely, a "send first" register 234 and a "send" register235.

The status register 231 includes a number of fields shown in FIG. 9A-2A.As described below in connection with FIG. 9B-1, each data routermessage injector port 223(l) and 223(r) includes a first-in first-outbuffer which buffers information from processor 200 from which thepacket 30 is generated. A send space field 240 identifies the amount ofspace left in the buffer in the particular port 223(l) or 223(r) that iscurrently selected to transmit the packet 30. The contents of the sendspace field 240 are provided by the currently selected left or rightdata router message injector port 223(l) or 223(r).

Two flags 241 and 242 indicate the status of the last reception andtransmission, respectively, of a data router message packet 30 throughthe currently selected port. If the last data router message packet 30to be received can be successfully received, flag 241 is set, and if thelast data router message packet 30 to be injected was successfullyinjected, flag 242 is set. The flags 241 and 242 are conditioned bymessage injector portion 220 and message ejector portion 221,respectively.

A receive message length field 243 indicates the length of the datarouter message packet 30 received through the currently selected port,and a length left field 244 identifies the amount of data in a datarouter message packet 30 currently being retrieved by the processor 200that is remaining to be retrieved. The contents of the receive messagelength field 243 correspond to the contents of length field 34 (FIG. 3)of the data router message packet 30. The contents of a receive tagfield 245 correspond to the contents of the tag field 35 of the samedata router message packet 30. The length left field 244 is effectivelyprovided by a counter into which the contents of length field 34 areloaded when the processor 200 begins retrieving the message packet 30,and which is decremented as the message data is transmitted to theprocessor 200. The contents of fields 243, 244 and 245 are provided bythe message ejector portion 221.

A send state field 246 and receive state field 247 identify the state ofinjection and ejection, respectively, of respective message packets 30by the message ejector portion 220 and message injector portion 221. Thesend state field 246, whose contents are provided by the messageinjector portion 220, indicates whether either or both of the left orright message injector ports 223(l) and 223(r) containpartially-injected data router message packets 30. Similarly, thereceive state field 247, whose contents are provided by the messageejector portion 221, indicates whether either or both of the left orright message ejector ports 225(l) and 225(r) contain partially-ejected(that is, received) data router message packets 30.

Finally, a router done flag 248, whose contents are actually provided bythe control network interface 204, indicates whether the router is emptyfollowing a message transfer operation. The condition of the router doneflag 248 is derived from the reduce operation performed over the controlnetwork 14 to determine whether the data router 15 is empty as describedabove.

The private register 232 also includes a number of fields, comprisingflags 250 through 256. Several flags, which are included in the ejectorcommon control/status portion 226, control the operation of the messageejector portion 221. A receive interrupt enable flag 250, when set,enables the data router interface 205 to generate an interrupt fortransmission by the network interface 202 to processor 200 when a datarouter message packet 30 is received by the currently selected left orright message ejector port 225(l) or 225(r). A receive stop flag 252,when set by the processor 200, disables reception of subsequent datarouter message packets 30 by the currently selected left or rightmessage ejector port 225(l) or 225(r). The currently selected port225(l) or 225(r) stops receiving flits immediately upon the flag 252being set. A receiver full flag 252, when set by the currently-selectedejector port 225(l) or 225(r), indicates that a buffer maintained by thecurrently-selected ejector port is full.

The private register 232 also includes a lock flag 251, included in theinjector common control/status portion 224, that controls the operationof the message injector portion 220. The lock flag 251 enables ordisables the currently selected left or right message injector port223(l) or 223(r). When set by processor 200, the currently selected leftor right message injector port 223(l) or 223(r) ignores subsequenttransmissions from processor 200, and the flag 242 in status register231 is cleared, indicating unsuccessful injection of the data routermessage packet 30.

The private register 232 also includes three flags that controloperation of the data router interface 205 in connection with theall-fall-down mode of the data router 15 as described above. A receivedall-fall-down flag 254, controlled by the control network interface 204,indicates that it has received a data router message packet 30 while thedata router 15 is operating in all-fall-down mode, for which the leaf 21is not the destination. An all-fall-down interrupt enable flag 255, whenset by processor 200, enables the network interface 202 to generate aninterrupt request for transmission to the processor upon the setting ofthe received all-fall-down flag 254. Finally, an all-fall-down enableflag 256, when set by processor 200, enables the control networkinterface 204 to set the all-fall-down bit 81 of the next controlnetwork message packet 60 that it transmits.

The remaining registers in the middle interface register set 230 areused to transmit and receive data router message packet information. Areceive register 233 contains a number of words 260(0) through 260(N)representing the data in a data router message packet 30 receivedthrough the currently selected left or right message ejector port 225(l)or 225(r). In reference numeral 260(N), "N" is an integer related to themaximum amount of data that can be transmitted in a single data routermessage packet 30. The data stored in receive register 233 is from thedata flits 36 of the received message packet 30. The receive register isrepresented by a single address in the address space of memory bus 203.The processor can retrieve the data from a message by iteratively usingthe address in a read operation over memory bus 203. It will beappreciated that the data router interface 205 decrements the contentsof the receive length left field 244 as the processor 200 accesses thereceive register to retrieve the message data.

Two registers, namely, the send first register 234 and the send register235 are provided to enable the processor to supply information used bythe message injector portion to generate data router message packets 30for injection into the data router 15. The send first register 234includes fields 270 and 271 in which message length and message taginformation is loaded. The contents of fields 270 and 271 are copiedinto the message length and message tag fields 34 and 35 in a datarouter message packet 30.

The send first register 234 also includes a message address field 273that is used to generate the contents of message address portion 31 ofpacket 30, and an address mode field 272. The message address in field273 can be an physical address, which specifically identifies the leaf21(y) to receive the message, or a relative address, which identifies adisplacement from the leaf 21(x) transmitting the data router messagepacket 30 to the leaf 21(y) to receive the packet 30. The contents ofthe address mode field 272 indicate whether the message address in field273 is an physical address or a relative address.

The send register 235, like receive register 233, contains a number ofwords 280(0) through 280(N) representing the data in a data routermessage packet 30 to be transmitted through the currently selected leftor right message injector port 223(l) or 223(r). In reference numeral280(N), "N" is an integer related to the maximum amount of data that canbe transmitted in a single data router message packet 30. The datastored in send register 235 is copied into the data flits 36 of thetransmitted message packet 30. The send register is represented by asingle address in the address space of memory bus 203. The processor canload data into the register by iteratively using the address in a writeoperation over memory bus 203.

As noted above, the processor 200 uses the data router interface middleregister set 230 when it does not specify the particular messageinjector port 223(l) or 223(r) to transmit a particular data routermessage packet 30. The data router interface 205 includes two additionalregister sets, identified as a left and right interface register sets290 and 291 (shown on FIG. 9A-2B), respectively, which the processor 200uses when specifies a left or right message injector port 223(l) or223(r) to transmit a particular data router message packet 30, or a leftor right message ejector port 225(l) or 225(r) from which it willretrieve data router message packet data. Both left and right interfaceregister sets 290 and 291 include respective status, private, receive,send first and send registers, identified by reference numerals 293-297(left register set 290) and 300-304 (right register set 291). Theregisters in register sets 290 and 291 have fields and flags that aresubstantially the same as those of respective registers 231-235 of themiddle interface register set, except that the left and right interfacestatus registers 293 and 300 do not have fields corresponding to sendand receive state fields 246 and 247 or router done flag 248 of statusregister 231. In addition, left and right interface private registers294 and 301 do not have fields corresponding to all-fall-down interruptenable flag 255 or all-fall-down enable flag 256 of private register232.

The data router interface 205 also includes a set of registers 292 whichcontain information that it uses, along with the message addressinformation in field 273 of the send first register 234 of the middleinterface register set 230 or corresponding fields of send firstregisters 296 or 303 of the respective left or right interface registerset, in generating address information for the message address field 31of a data router message packet 30 to be transmitted. As describedabove, the system 10 can be partitioned, and a partition base register305 and partition size register 306 contain values identifying the baseand size of the processing element's partition. In particular, thepartition base register 305 contains the index (i) of the leaf 21(i) inthe system that is the lowest-indexed element in the partition. Inaddition, the contents of the partition size register 306 identify thenumber of leaves 21 in the partition. A physical self address register312 for a particular leaf 21(i) identifies the leaf's own index "i" inthe system 10, which comprises an address or other identifier thatuniquely identifies the leaf 21 in the system.

The address information register set 292 also includes a set ofregisters 307 and 310-311 which are used with a chunk table 327 (whichis described in more detail below in connection with FIGS. 9B-1 and9B-2) and that provides additional addressing information used ingenerating address information for the message address field 31 of adata router message packet 30 to be transmitted. The chunk table ismaintained by the injector common control/status portion 224 of theinjector portion 220. The chunk table permits groups of leaves 21(i) tobe to be substituted for other groups, without requiring the addressprovided by the processor 200 to reflect the substitution. This mayfacilitate, for example, one or more groups of leaves 21(i) being madeinaccessible, or "mapped out," of the data router 15, which may behelpful if one or more leaves 21(i) in the group is defective. The chunktable contains information that is used in generating a portion of themessage address for a data router message packet 30, permitting amessage packet to be re-directed from the leaf 21(i) in the originalgroup to the leaf 21(i) of a group that is assigned as a substitutegroup.

To accommodate that operation, as described below in more detail inconnection with FIG. 9B-2, the chunk table includes a memory including aplurality of entries. Each entry comprises a pointer, or bits comprisinga portion of the address to be used as a substitute. If the address fromthe processor 200, in the particular send first register 234, 296 or303, is a relative address, a portion of it is used to access the chunktable, to obtain the portion to be used in generating the address forthe message address portion of the data router message packet 30. Theparticular portion of the relative address used to address the chunktable depends on the size of the group to be mapped in and out.

A chunk table address register 307 and chunk table data register 310 areused together to enable loading of the entries in the chunk table. Thechunk table address in register 307 is used to identify the entry in thechunk table into which the contents of the chunk table data register 310will be stored. The contents of the chunk size register 311 identify thenumber of leaves 21(i) in a group which may be mapped in or out, which,in turn, determines the particular bits of the relative address to beused in accessing the chunk table 327.

Finally, the registers maintained by the data router interface 205include the previously-mentioned data router message counter 313. Datarouter message counter 313 is maintained by the injector/ejector commoncontrol/status portion 222. The message counter 313 is incremented toreflect the injection by data router message injector port 220 of a datarouter message packet over the data router 15 during a message transferoperation, and decremented to reflect the ejection, by the data routermessage ejector port 221 of a data router message packet 30 that itreceives from the data router 15. The injector/ejector commoncontrol/status portion 222 generates a CUR MSG CNT current message countsignal which identifies the current value of the message counter 313,and which it provides to the control network interface 204 for use ingenerating a router done control network message as described above.

ii. Message Injector Portion 220

With this background, details of circuits comprising the left and rightmessage injector ports 223(l) and 223(r), along with portions of theinjector common control/status portion 224, in the message injectorportion 220 will be described in connection with FIGS. 9B-1 through9B-8. Similarly, details of circuits comprising the left and rightmessage ejector ports 225(l) and 225(r), along with portions of theejector common control/status portion 226, in the message ejectorportion 221 will be described in connection with FIGS. 9C-A through9C-7. In addition, details of circuits comprising the injector/ejectorcommon control/status portion 222 will be described in connection withFIGS. 9D-1 through 9D-7.

Since the circuits comprising the left and right message injector ports223(l) and 223(r) are substantially similar, only one (without referenceto it being the left or right port) will be described in connection withFIGS. 9B-1 through 9B-8. FIG. 9B-1 depicts a general block diagram of amessage injector port 223. With reference to FIG. 9B-1, the messageinjector port 223 includes a series of stages that receive informationfrom the processor 200 to be used in generating the message packet 30 tobe injected, buffer the information, perform any necessary addressgeneration, divide the information into flits and, under control of theIN FLY injector fly signal, transmit the flits as successive IN FLITinjected flit signals. In the following description, the informationreceived from the processor 200 is in the form of words each having, forexample, thirty two bits, and each flit comprises four bits transmittedin parallel.

Information from the processor 200, received by the processing elementinterface 212, is first loaded into a write stage 320. The write stage320 is connected to the processing element interface bus 211, is loadedin response to LD CTRL load control signals from the injector commoncontrol/status portion 224. The write stage 320 operates as the input toan injector first-in first-out buffer (FIFO) 321. When the processor 200loads information into the send first registers 234, 296 and 303, or thesend registers 235, 297 and 304, it is essentially received by the writestage 320.

If the information from the processor 200 is addressed to a send firstregister, the write stage 320 generates a thirty-four bit address wordin which the low-order twenty bits comprise the address information, themiddle bits comprise message packet length and tag information and thehigh-order bits comprise two address mode bits which it derives from theaddress mode field 272. In particular, if one of the address mode bitsis set, the address information is an physical address, and if the otheraddress mode bit is set the address information is a relative address.In response to the successive words thereafter addressed by theprocessor 200 to the send register, the write stage 320 generatessuccessive thirty-four bit words in which the two high-order addressmode bits have the value zero, with the thirty-two bits data appearingin the low-order portion. It will be appreciated that the zero addressmode bits effectively identify the low-order thirty-two bits ascomprising data.

The output of write stage 320, INJ FIFO DATA (33:0) injector first-infirst-out buffer data signals defining thirty-four bit words, arecoupled to the data input terminals of FIFO 321. The FIFO 321 storessuccessive words received thereby in thirty-four bit storage locations,thereby accommodating the thirty-four bit address words. The FIFO 321loads INJ FIFO DATA (33:0) signals in response to FIFO LD EN load enablesignals from the injector common control/status portion 224 and the NODECLK signal. In addition, FIFO 321 generates FIFO STATUS status signalsindicating whether it can accept additional words from the write state320. The injector common control/status portion 224 can use the FIFOSTATUS signals in controlling the operation of the write stage 320 andflow of information, including addresses and data, to FIFO 321.

In addition, the injector FIFO 321 receives a FRAME signal and a FLUSHsignal from the injector common control/status portion 224. The FRAMEsignal, when asserted, indicates that address and data words for anentire data router message packet 30 have been loaded into the injectorFIFO 321. At that point, the message injector port 223 will generate adata router message packet 30 using the words. The FLUSH signal, whenasserted, indicates that address and data words being loaded into theinjector FIFO 321 are to be flushed, and that no data router messagepacket 30 is to be generated in response to those words. The injectorcommon control/status portion 224 may assert the FLUSH signal, forexample, if it detects an error in connection with receipt ofinformation for the data router message packet 30 as providd by theprocessor 200.

Essentially, the injector FIFO 321 includes a plurality of storagelocations that are sequentially loaded with INJ FIFO DATA signalsdefining a word from the write stage 320. In addition the injector FIFO321 includes a store pointer to the next location to store a word, and amessage pointer pointing to the first word for a data router messagepacket 30. If the injector common control/status portion 224 asserts theFRAME signal, the message pointer is advanced to point to the samelocation as the store pointer. In addition, the injector FIFO 321 willassert a MSG AVAIL message available signal. On the other hand, if theinjector common control/status portion 224 asserts the FLUSH signal, thestore pointer is moved back to point to the same location as the messagepointer, so that the previously-written words for the flushed datarouter message packet 30 can be overwritten.

In addition, the injector FIFO 321 includes a read pointer that pointsto the next location to be read, and as long as the read pointer has notadvanced to point to the same location as the message pointer, theinjector FIFO 321 asserts a FIFO NE not empty signal.

The FIFO 321 effectively operates as a buffer between processor 200 andthe data router 15. As indicated above, when the message ejector port223 begins transmitting a data router message packet 30, the data routernode 21(1,j,k) connected thereto expects to receive successive flits inthe packet 30 in synchrony with successive ticks of the SYS CLK systemclock signal, as long as the node 21(1,j,k) maintains the IN FLYinjector fly signal asserted. Thus, it will be appreciated thattransfers from the FIFO 321, which are directly controlled by a RD FIFOread FIFO signal from an output latch 322 connected to the output ofFIFO 321, are effectively controlled by the IN FLY injector fly signal,and by the conditions of stages in the message injector port after theFIFO 321. These stages include the output latch 322, an physical addresscomputation stage 323, a physical address computation stage 324, amessage address computation stage 325 and a transmitter stage 326. Thetransmitter stage 326, while the data router node 22(1,j,k) is assertingthe IN FLY injector fly signal, receives the actual IN FLIT injectorflit signals that control transmission of the successive flits of thedata router message packet 30.

In any event, the FIFO 321, in response to the RD FIFO read FIFO signalfrom the output latch 322, transmits INJ FIFO OUT (33:0) injector FIFOoutput signals defining a word. In response to an ADV AACS advanceabsolute address computation stage signal from the absolute addresscomputation stage 323, which indicates that the stage 323 can acceptanother word from the FIFO 321, and the FIFO NE not empty signal fromthe FIFO 321, and in synchrony with the NODE CLK node clock signal, theoutput latch 322 latches the INJ FIFO OUT injector FIFO output signalsfrom FIFO 321. The output latch thereupon transmits the latched signalsto absolute address computation stage 323 as LAT FIFO OUT (33:0) latchedFIFO output signals. The output latch 322 also asserts the RD FIFO readFIFO signal to enable the FIFO 321 to transmit to it INJ FIFO OUTinjector first-in first-out buffer output signals defining the next wordstored in the FIFO 321.

The absolute address computation stage 323 receives the LAT FIFO OUT(33:0) latched FIFO output signals from the output latch 322, determineswhether the signals comprise data, an physical address or a relativeaddress, and if the signals comprise a relative address determine anphysical address value. The absolute address computation stage 323, inresponse to an ADV PACS advance physical address computation stagesignal from the physical address computation stage 324 and the NODE CLKnode clock signal, transmits ABS ADRS/DATA OUT (33:0) absolute addressor data output signals to the physical address computation stage 324.The ABS ADRS/DATA OUT (33:0) absolute address or data output signalscomprise an physical address or a physical address if the LAT FIFO OUT(33:0) latched FIFO output signals received from the output latch 322included an address, and data if the LAT FIFO OUT (33:0) latched FIFOoutput signals comprised data.

If the LAT FIFO OUT (33:0) latched FIFO output signals comprise arelative address, the absolute address computation stage 323 uses thechunk table 327 and the chunk size register 311, maintained by theinjector common control/status portion 224 (FIG. 9A-1), in determiningthe physical address. As will be described in more detail below inconnection with FIG. 9B-2, the absolute address computation stage 327generates a CHUNK TABLE OFFSET (5:0) signals to identify a location inthe chunk table 327. In response, the chunk table 327 transmits to thephysical address computation stage 323 the contents of the identifiedlocation as CT ADRS chunk table address signals. The absolute addresscomputation stage 323 uses the CT ADRS chunk table address signals ingenerating the physical address. The absolute address computation stage323 uses the contents of the chunk size register 311 in generating theCHUNK TABLE OFFSET (5:0) signals, and in using the CT ADRS chunk tableaddress signals in generating the physical address.

The physical address generated by absolute address computation stage 323is the address of the destination leaf 21(y) relative to the beginningof the partition including the source leaf 21(x). The physical addresscomputation stage 324 uses the physical address to determine a physicaladdress, which is the address of the destination leaf 21(y) relative tothe first leaf 21(0) in the system 10. In this operation, the stage 324uses the contents of the partition base register 305. The physicaladdress computation stage 324, in response to a RUN signal from thetransmitter stage 326 and in synchrony with the NODE CLK signal,generates PHYS ADRS/DATA OUT (33:0) physical address or data outputsignals. The PHYS ADRS/DATA OUT (33:0) physical address or data outputsignals comprise the physical address if the ABS ADRS/DATA OUT (33:0)absolute address or data output signals comprised an address, or data ifthe ABS ADRS/DATA OUT (33:0) signals comprised data.

The message address computation stage 325 receives the PHYS ADRS/DATAOUT (33:0) physical address or data output signals and generates anaddress for inclusion in the message address portion 31 of the datarouter message packet 30. The message address computation stage 325 usesthe contents of the physical self address register 312 in thisoperation. The message address computation stage 325, also in responseto the RUN signal from the transmitter stage 326 and in synchrony withthe NODE CLK signal, generates MSG OUT (31:0) message out signals. TheMSG OUT (31:0) message out signals comprise the message address if thePHYS ADRS/DATA OUT (33:0) physical address or data output signalscomprise a physical address, and data if the PHYS ADRS/DATA OUT (33:0)signals comprise data. The high-order MSG OUT (32) signal identifieswhether the remaining MSG OUT (31:0) signals represent a message addressor data.

Finally, the transmitter stage 326 receives the MSG OUT (32:0) messageout signals and generates therefrom the IN FLIT inject flit signals,comprising successive four-bit nibbles from the MSG OUT (31:0) signals.The transmitter stage 326 transmits successive nibbles in synchrony withthe NODE CLK node clock signal while the IN FLY injector fly signal isasserted. In addition, the transmitter stage 326 uses the IN FLYinjector fly signal in generating the RUN signal that controls themessage address computation stage 325 and the physical addresscomputation stage 324.

The details of various stages of address computation performed by stages323 through 325 and of generating successive nibbles for transmission asIN FLIT input flit signals will be described in connection with FIGS.9B-2 through 9B-8. FIG. 9B-2 depicts details of the absolute addresscomputation stage 323. With reference to FIG. 9B-2, the absolute addresscomputation stage 323 receives the LAT FIFO OUT (33:0) latched FIFO outsignals from the output latch 322. The two high-order signals, namely,the LAT FIFO OUT (33:32) signals, which identify whether the LAT FIFOOUT (31:0) signals comprise and address and, if so, the address mode,are coupled to a decoder 330 and one input of a multiplexer 331. If theLAT FIFO OUT (33:32) signals identify the relative address mode, thedecoder 330 asserts a REL relative signal.

The absolute address computation stage includes an address/data latch332 that latches the physical address or data, which are derived fromthe LAT FIFO OUT (31:0) latched FIFO output signals. The latch 332latches three groups of signals in unison in response to the ADV AACSadvance absolute address computation stage and NODE CLK node clocksignals. One group, the LAT FIFO OUT (31:20) signals, are coupleddirectly to the latch 332 from output latch 322. In one particularembodiment, if the low-order LAT FIFO OUT (19:0) signals compriseaddress signals, the LAT FIFO OUT (31:20) signals include the length andtag information, and thus would be invariant in the computation. Inaddition, a second group of signals latched by latch 332, namely, thelow order LAT FIFO OUT (1:0) latched FIFO output signals, comprising thetwo low-order address signals, are also invariant in the computation,and thus are coupled directly to the latch 332.

The third group of signals latched by latch 332 are not invariant in thecomputation of the physical address. The LAT FIFO OUT (19:2) signals arecoupled to one input terminal of a multiplexer 333, which is controlledby the REL relative signal from decoder 330. If the REL relative signalis negated, which will occur if the LAT FIFO OUT (31:0) signals comprisean physical address or data, the multiplexer couples the LAT FIFO OUT(19:2) signals directly to the latch 332. When the ADV PACS advancephysical address computation stage signal is asserted, an OR gate 340 isenergized to assert an ADV EN advance enable signal, which, in turn,energizes one input terminal of an AND gate 344.

If the REL relative signal is negated, an inverter 339 energizes an ORgate 343, which enables the other input terminal of AND gate 344,enabling it to assert the ADV AACS advance absolute address computationstage signal. The ADV AACS signal enables latch 332, which, in responseto the next tick of the NODE CLK node clock signal, latches the LAT FIFOOUT (31:20) signals, the output of multiplexer 333, which corresponds tothe LAT FIFO OUT (19:2) signals, and the LAT FIFO OUT (1:0) signals.Accordingly, the latch 332 will latch the entire physical address ordata provided by the output latch 322. The latch 332 transmits thelatched signals to the physical address computation stage as the ABSADRS/DATA OUT (31:0) absolute address data out signals.

Contemporaneously, an address mode latch 338 will latch the output ofmultiplexer 331. The multiplexer 331 selectively couples either the LATFIFO OUT (33:32) address mode signals or NOP (1:0) null operationsignals to the input terminals of the address mode latch 338 in responseto a LD NOP MOD load null operation mode signal from an AND gate 345. Aswill be described below in more detail, the NOP (1:0) null operationsignals comprise a code that identifies an error condition in connectionwith conversion from a relative address to an physical address. In thiscase, since the REL relative signal is negated, the AND gate 345 isde-energized and the LD NOP MOD load null operation mode signal isnegated, enabling the multiplexer 331 to couple the LAT FIFO OUT (33:32)signals to the mode latch 338. The mode latch 338 transmits the latchedsignals to the physical address computation stage as the ABS ADRS/DATAOUT (33:32) absolute address/data out signals.

On the other hand, if the REL relative signal is asserted, which willoccur if the LAT FIFO OUT (31:0) signals comprise a relative address,the multiplexer 333 couples CONV ABS ADRS (19:2) converted absoluteaddress signals to the latch 332. The CONV ABS ADRS (19:2) convertedabsolute address signals are provided by a conversion circuit comprisinga window extractor 334, the chunk table 327, a window inserter 335 and awindow identifier 336.

The window extractor 334 receives the LAT FIFO OUT (17:2) signals andWIN SEL window select signals and generates, in response thereto, theCHUNK TABLE OFFSET (5:0) signals to address the chunk table 327. The WINSEL window select signals identify a set of six consecutive signals inthe LAT FIFO OUT (17:2) signals for the window extractor to couple tothe chunk table 327 as the CHUNK TABLE OFFSET (5:0) signals.

As noted above, the contents of chunk size register 311 identify thelevel "i" in the data router 15 at which each data router node group20(i,j) defines a chunk. In one particular embodiment, the lowest levelin the data router 15 at which a chunk may be defined is level four,which has at least two-hundred and fifty six consecutive leaves 21(i)each identified by a data router address. In that embodiment, each chunkis defined by the encoding of the LAT FIFO OUT (7:2) signals. In thesame embodiment, the highest level at which a chunk may be defined islevel nine, which has at least 256k (k=1024) consecutive leaves. In thatcase, each chunk is defined by the encoding of the LAT FIFO OUT (17:12)signals.

As the chunk size, as indicated by the value stored in the chunk sizeregister 311, increases, higher order ones of the LAT FIFO OUT (19:2)signals are used to access the chunk table 327. The window identifier336 decodes the contents of the chunk size register to identify thebeginning of six sequential ones of the LAT FIFO OUT (17:2) signals toaddress the chunk table 327. The WIN SEL window select signals enablethe window extractor 334 to select the particular ones of the LAT FIFOOUT (19:2) signals to address the chunk table 327, and transmit them tothe chunk table 327 as the CHUNK TABLE OFFSET (5:0) signals. If, forexample, the chunk size register 311 identifies the chunk size as beingtwo-hundred and fifty six, the window extractor 334 selects the LAT FIFOOUT (7:2) signals, whereas if it identifies the chunk size as being 256k(k=1024) the window extractor 334 selects the LAT FIFO OUT (17:12)signals. For window sizes between two hundred and fifty six and 256k,incrementing by powers of "four," the window extractor selects sixconsecutive ones of the LAT FIFO OUT (9:4), LAT FIFO OUT (11:6), LATFIFO OUT (13:10), and LAT FIFO OUT (15:12) signals, respectively.

When the physical address computation stage 324 asserts the ADV PACSadvance physical address computation stage signal, the signal energizesan OR gate 340, which asserts an ADV EN advance enable signal. The ADVEN advance enable signal enables one input terminal of an AND gate 341.If the decoder 330 is asserting the REL relative signal, indicating thatthe LAT FIFO OUT (31:0) signals represent a relative address, the ANDgate 341 is energized to assert a READ CT read chunk table signal, whichis coupled to a read enable terminal of the chunk table 327. In responseto the asserted READ CT read chunk table signal, the chunk table 327transmits the contents of the location identified by the CHUNK TABLEOFFSET (5:0) signals as CT ADRS (5:0) chunk table address signals to thewindow inserter 335, and asserts a CT VALID chunk table valid signal.

The window inserter 335 essentially performs the reverse operation asthe window extractor. The window inserter substitutes the six-bit CTADRS (5:0) chunk table address signals into the LAT FIFO OUT (19:2)signals, into six sequential bit locations represented by the LAT FIFOOUT (19:2) signals, and couples the result as CONV ABS ADRS (19:2)converted absolute address signals to the second input terminal ofmultiplexer 333. The window inserter substitutes the CT ADRS (5:0) chunktable address signals for the same order signals as were extracted bythe window extractor 334. The multiplexer 333, under control of theasserted REL relative signal from decoder 330, couples the CONV ABS ADRS(19:2) converted absolute address signals to the input terminal of latch332.

The CT VALID chunk table valid signal, when asserted, indicates that theCT ADRS (5:0) chunk table address signals represent a valid value. Inone embodiment, delays of several ticks of the NODE CLK signal may berequired between the time the CHUNK TABLE OFFSET signals are coupled tothe input terminals of the chunk table and the READ CT read chunk tablesignal is asserted, and the time the chunk table 327 provides valid CTADRS (5:0) signals. During that time, signals may nonetheless be latchedin the latches 332 and 338. The CT VALID signal, if negated, essentiallyenables signals representing a null operation code to be latched in themode latch 338, which can be used by subsequent stages, if they progressthereto, to indicate that the latched signals should be ignored. It willbe appreciated that, since the chunk table 327 is only used inconnection with relative addresses, if the address is a physicaladdress, or if the LAT FIFO OUT (33:0) signals represent data, the CTVALID signal will be ignored and the null operation code will not beused.

More specifically, if the chunk table 327 transmits a negated CT VALIDchunk table valid signal, an inverter 346 enables an input terminal ofAND gate 345. If (a) the REL signal is asserted, indicating that the LATFIFO OUT (33:0) signals represent a relative address, and (b) an ADVPACS signal is asserted, which in turn enables an OR gate 340 to assertan ADV EN advance enable signal, the AND gate 345 is energized to assertthe LD NOP MOD load null operation mode signal. The asserted LD NOP MODsignal enables the multiplexer 331 to couple NOP null operation signalsto the input terminal of mode latch 338, which latches them instead ofthe LAT FIFO OUT (33:32) signals. AND gate 345 remains asserted untilthe CT VALID signal is asserted.

Contemporaneously, an address mode latch 338 will latch the output ofmultiplexer 331. The multiplexer 331 selectively couples either the LATFIFO OUT (33:32) address mode signals or NOP (1:0) null operationsignals to the address mode latch 338 in response to the LD NOP MOD loadnull operation mode signal from AND gate 345. Since the LD NOP MODsignal is asserted, the multiplexer 331 couples NOP null operationsignals to the mode latch 338. If the absolut address computation stage323 is asserting the ADV AACS advance enable signal, the mode latch 338latches the coupled NOP signals at the next tick of the NODE CLK signal.The mode latch 338 transmits the latched signals to the physical addresscomputation stage as the ABS ADRS/DATA OUT (33:32) absolute address/dataout signals.

On the other hand, when the chunk table 327 transmits an asserted CTVALID signal, inverter 346 disables AND agate 345 to negate the LD NOPMOD signal. In addition, the asserted CT VALID signal enables one inputterminal of an AND gate 342. The assertion of the REL relative signalenables the other input terminal of AND gate 342, which is energized toassert a REL ADV relative advance signal. The REL ADV relative advancesignal energizes an OR gate 343, which, in turn, enables one inputterminal of an AND gate 344. Since the asserted ADV EN advance enablesignal is enabling the other input terminal of AND gate 344, the ANDgate 344 is energized, thereby asserting an ADV AACS advance absoluteaddress computation stage signal. The asserted ADV AACS signal enablesthe latch 332, which latches the LAT FIFO OUT (31:20), CONV ABS ADRS(19:2) and LAT FIFO OUT (1:0) signals at the next tick of the NODE CLKnode clock signal. The latch 332 transmits the latched signals to thephysical address computation stage as the ABS ADRS/DATA OUT (31:0)absolute address data out signals.

It will be appreciated that, since the ADV AACS advance absolute addresscomputation stage signal is asserted, the output latch 322 will latchthe next word from the injector FIFO 321. Since the next word will bethe next word of the message, not an address word, the LAT FIFO OUT(33:32) signals will not indicate that the LAT FIFO OUT (31:0) signalsrepresent a relative address. In that case the decoder 330 will negatethe REL relative signal, enabling the multiplexer 333 to couple the LATFIFO OUT (19:2) signals to the input terminal of address/data latch 332.

The negated REL relative signal also enables an inverter 339 to energizeOR gate 343, which, in turn, enables one input terminal of AND gate 344.If the ABS ADR NOP absolute address null operation signal fromcomparator 347 is not asserted, when the physical address computationstage 324 next asserts the ADV PACS advance physical address computationstate signal, OR gate 340 is energized. In response the AND gate 344asserts the ADV AACS advance absolute address computation stage signal,enabling the address/data latch 332 to latch the LAT FIFO OUT (31:0)signals at the next tick of the NODE CLK signal.

In addition, since the REL relative signal is negated, AND gate 345maintains the LD NOP MOD load null operation signal negated, therebyenabling the multiplexer 331 to couple the LAT FIFO OUT (33:32) signalsto the input terminal of mode latch 338. The mode latch 338 latchesthese signals contemporaneously with the latching by the address/datalatch 332 of the LAT FIFO OUT (31:0) signals. It will be appreciatedthat these operations will occur iteratively with each tick of the NODECLK signal, while the ADV PACS advance physical address computationstage signal is set, enabling the data words to be iteratively latchedin the latches 332 and 338 and thereby transmitted through the absoluteaddress computation stage.

The absolute address computation stage 323 includes a comparator 342that compares the ABS ADRS/DATA OUT (33:32) absolute address/data outsignals to SND ADRS NOP send address null operation signals. If themultiplexer 331 coupled the NOP null operation signals to be latched bythe mode latch 338, the comparator will assert an ABS ADR NOP absoluteaddress null operation signal, which enables the OR gate 340 to assertthe ADV EN advance enable signal. Since the REL relative signal isnegated, inverter 339, through OR gate 343 enables one input terminal ofAND gate 344, which is energized to assert the ADV AACS advance absoluteaddress computation stage signal.

As described above, the ADV AACS signal enables the latches 332 and 338to latch the next word defined by the LAT FIFO OUT (33:0) signalsprovided by the output latch 322 at the next tick of the NODE CLKsignal, and enables the output latch 322 to supply the next word in theFIFO 321. This occurs iteratively with successive ticks of the NODE CLKnode clock signal until the next address word is latched in latches 332and 338, in which case the ABS ADR NOP absolute address null operationsignal is negated. At that point, control over the sequencing throughthe absolute address computation stage 323 returns to the ADV PACSadvance physical address computation stage signal from the physicaladdress computation stage 324.

As noted above, the absolute address computation stage 323 transmits ABSADRS/DATA OUT (33:0) absolute address/data output signals, in the formof successive words, to the physical address computation stage 324. Inthis operation, if the ABS ADRS/DATA OUT (31:0) signals represent anaddress derived from signals representing a relative address, thephysical address is obtained by adding the address value to the value inthe partition base register 305. If the ABS ADRS/DATA OUT (31:0) signalsrepresent an address derived from signals representing an physicaladdress, the address from the absolute address computation stage is thephysical address. In either case, the address mode, as indicated by thehigh order ABS ADRS/DATA OUT (33:32) signals, control the operations ofthe physical address computation stage. FIG. 9B-3 depicts a detailedblock diagram of the physical address computation stage 324.

With reference to FIG. 9B-3, the physical address computation stage 324receives the ABS ADRS/DATA OUT (33:0) signals and couples the two highorder ABS ADRS/DATA OUT (33:32) signals to the input terminal of adecoder 350 and one input terminal of a mode latch 351. At this point,if the ABS ADRS/DATA OUT (31:0) signals represent an address, the twohigh-order ABS ADRS/DATA OUT (33:32) signals continue to indicate theaddress mode of the original address. If the ABS ADRS/DATA OUT (33:32)signals identify the relative address mode, decoder 350 generates a RELADRS MODE relative address mode signal.

Of the remaining ABS ADRS/DATA OUT (31:0) absolute address/data outsignals, the high-order ABS ADRS/DATA OUT (31:20) signals are coupleddirectly to the input terminal of a physical address latch 352. As notedabove, if the ABS ADRS/DATA OUT (31:0) signals represent an address, thehigh-order ABS ADRS/DATA OUT (31:0) signals carry the message length andtag signals for inclusion in fields 34 and 35 of a data router messagepacket 30, and thus are invariant under conversion. The remaining ABSADRS/DATA OUT (19:0) signals, comprising the address are coupled to oneinput terminal of a multiplexer 353 and one input terminal of an adder354. The second input terminal of adder 354 is provided by the partitionbase register 305.

The multiplexer 353 is controlled by the REL ADRS MODE relative addressmode signal from the decoder 350. If the REL ADRS MODE relative addressmode signal is negated, indicating that the ABS ADRS/DATA OUT (19:0)signals represent either an physical address or data, the multiplexer353 couples them directly to the physical address latch 352. If thetransmitter stage 326 is asserting the RUN signal, an OR gate 355asserts the ADV PACS advance physical address computation stage signal,enabling the physical address latch 352 and the mode latch 352 to latchthe signals at their respective input terminals in synchrony with thenext tick of the NODE CLK signal. In this case, the physical addresslatch 352 will latch the ABS ADRS/DATA OUT (31:20) signals directly fromthe absolute address computation stage, and the ABS ADRS/DATA OUT (19:0)signals coupled thereto by the multiplexer 353.

On the other hand, if the decoder 350 is asserting the REL ADRS MODErelative address mode signal, indicating that the ABS ADRS/DATA OUT(31:0) signals are derived from a relative address, the multiplexer 353couples signals output from the adder 354 to the physical address latch352. These signals represent a value comprising the sum of the addressdefined by the current ABS ADRS/DATA OUT (19:0) absolute address/dataoutput signals and the contents of the partition base register. If theOR gate 355 is asserting ADV PACS advance physical address computationstage signal, the physical address latch 352 latches these signals,along with the ABS ADRS/DATA OUT (31:0) signals, in synchrony with thenext tick of the NODE CLK signal. The mode latch 351 provides thehigh-order PHYS ADRS/DATA OUT (33:32) physical address/data outputsignals, and the physical address latch 352 provides the low order PHYSADRS/DATA OUT (31:0) signals, both of which are coupled to the messageaddress computation stage 325.

The high-order PHYS ADRS/DATA OUT (33:32) physical address/data outputsignals are also coupled to an input terminal of a comparator 356. Thecomparator 356 also receives the SND ADRS NOP send address nulloperation signals. If the PHYS ADRS/DATA OUT (33:32) signals correspondto the SND ADRS NOP send address null operation signals, the comparator351 generates a PHYS ADRS NOP physical address null operation signal. Itwill be appreciated that the comparator 356 asserts the PHYS ADRS NOPsignal under the same circumstance as the comparator 347 asserts the ABSADRS NOP absolute address null operation signal. That is, the comparator356 asserts the PHYS ADRS NOP signal in response to the coincidence of(i) the ABS ADRS/DATA OUT (19:0) signals representing a relative addressand (ii) the chunk table location 327 used by the absolute addresscomputation stage 323 (FIG. 9B-2) was not valid.

The asserted PHYS ADRS NOP physical address null operation signal alsoenables the physical address computation stage to perform operationssimilar to that performed by the absolute address computation stage inresponse to the asserted ABS ADRS NOP signal, that is, sequencingthrough words of a message provided thereto by the absolute addresscomputation stage until it receives a valid word. The asserted PHYS ADRSNOP physical address null operation signal enables the OR gate 355 tomaintain the ADV PACS advance physical address computation stage signalasserted, enabling the latches 351 and 352 to iteratively latch thesignals provided thereto in response to successive ticks of the NODE CLKsignal, until the comparator 356 determines that the contents of themode latch 351 are associated with a valid address word.

The PHYS ADRS/DATA OUT (33:0) physical address data out signals arecoupled to the message address computation stage 325. If the high-orderPHYS ADRS/DATA OUT (33:32) signals indicate that the remaining PHYSADRS/DATA OUT (31:0) signals correspond to an address, the messageaddress computation stage 325 (i) determines the contents of the headerfield 40, (ii) arranges the first thirty-two bit message word, whichincludes the header, the down path identification portion 41, themessage length field 34 and the message tag field 35, and (iii) latchesthe message word for transmission to the transmitter stage. If the PHYSADRS/DATA OUT (33:32) signals indicate that the remaining PHYS ADRS/DATAOUT (31:0) signals do not comprise an address, they comprise data thatare latched for transmission to the transmitter stage.

A detailed block diagram of the message address computation stage isdepicted in FIG. 9B-4. With reference to 9B-4, the high-order PHYSADRS/DATA OUT (33:32) physical address/data out signals are coupled totwo decoders 360 and 316. If the PHYS ADRS/DATA OUT (33:32) signalsindicate that the associated low order PHYS ADRS/DATA OUT (31:0) signalsare derived from an address word including a physical address, decoder360 asserts an IS PHY ADRS signal. On the other hand, if the PHYSADRS/DATA OUT (33:32) signals indicate that the associated low orderPHYS ADRS/DATA OUT (31:0) signals are derived from an address wordincluding a relative address, decoder 360 asserts an IS REL ADRS signal.Both the IS PHYS ADRS signal and IS REL ADRS signal are coupled to inputterminals of an OR gate 362, which generates an asserted IS ADRS isaddress signal when either the IS PHYS ADRS signal or the IS REL ADRSsignal is asserted. Thus, the IS ADRS signal will be asserted if thePHYS ADRS/DATA OUT (31:0) signals represent an address word.

Selected ones of the low-order PHYS ADRS/DATA OUT (31:0) physicaladdress/data out signals are coupled to respective input terminals ofthree multiplexers 363, 364, and 365. In particular, the PHYS ADRS/DATAOUT (31:8) signals are coupled to one data input terminal of multiplexer363, the PHYS ADRS/DATA OUT (7:4) signals are coupled to thecorresponding data input terminal of multiplexer 364, and the PHYSADRS/DATA OUT (3:0) signals are coupled to the corresponding data inputterminal of multiplexer 365. If the IS ADRS signal is negated, whichoccurs if the PHYS ADRS/DATA OUT (31:0) signals represent data, themultiplexers 363, 364 and 365 couple the signals at these data inputterminals through their respective output terminals to the inputterminal of a staging register 366. If the transmitter stage 326 isasserting the RUN signal, the staging register latches these signals inresponse to the next tick of the NODE CLK signal. The latched signalsare coupled to the transmitter stage 326 as the MSG OUT (31:0) signals.

The second data input terminals of multiplexers 364 and 365 are coupledto PHYS ADRS/DATA OUT (27:24) signals and PHYS ADRS/DATA OUT (23:20)signals respectively. These signals represent the message length andmessage tag, which are inserted into fields 34 and 35, respectively, ofthe data router message packet 30 (FIG. 3). The second data inputterminal of multiplexer 363 is coupled to a message address portiongenerating circuit 367 which, in response to the PHYS ADRS/DATA OUTsignals (19:0), generates MSG ADRS (23:0) signals, which represent theheader 40 and down path identification portion 41 of the message addressportion 31 of the data router message packet 30 to be transmitted. Whenthe OR gate 362 asserts the IS ADRS signal, multiplexers 363, 364 and365 couple the signals at their second data input terminals to the inputterminals of staging register 366. If the transmitter stage 326 isasserting the RUN signal, the staging register latches these signals inresponse to the next tick of the NODE CLK signal. In particular, thestaging register 366 latches the message address portion in its bitsthirty-one through eight, the length in bits seven through four, and thetag in bits three through zero. The latched signals are coupled to thetransmitter stage 326 as the MSG OUT (31:0) signals.

In addition, if the transmitter stage 326 is asserting the RUN signal, aregister 368 latches the IS ADRS is address signal in response to thenext tick of the NODE CLK signal. The latched signal is coupled to thetransmitter stage 326 as the MSG OUT (32:0) signal.

The general block diagram of the message address portion generatingcircuit 367 is depicted in FIG. 9B-4, and logic diagrams of severalcircuits therein are depicted in FIGS. 9B-4 through 9B-7. The messageaddress portion generating circuit 367 includes an exclusive-OR ("XOR")gate 370, which receives the PHYS ADRS/DATA OUT (19:0) signals and thecontents of physical self address register 312, and generates REL ADRS(19:0) relative address signals which comprises the bit-wiseexclusive-OR of the input signals. That is, for example, the XOR gate370 generates the REL ADRS (19) signal as the exclusive-OR of the PHYSADRS/DATA OUT (19) signal and bit nineteen from the physical selfaddress register 312. The REL ADRS (19:0) relative address signals havea binary-encoded value that represents the displacement from the sourceleaf 21(x) to the destination leaf 21(y).

The REL ADRS (19:0) relative address signals are coupled to inputterminals of a header nibble calculation logic circuit 371 and a headernibble select logic circuit 372. The header nibble calculation logic 371(FIG. 9B-5) uses the REL ADRS (19:0) signals to determine the level indata router 15 to which the data router message packet 30 must be sentto reach its destination. As described above, the height is the level"i" of the first data router node group 20(i,j) which the data routernode packet 30 reaches in the upward path through the data router 15that is the root of a sub-tree that includes both the source leaf 21(x)and destination leaf 21(y). The header nibble calculation logic 371generates a four-bit nibble, comprising HDR (3:0) header signals, thatis binary-encoded to identify this level.

The header nibble select logic circuit 372 (FIG. 9B-6) determines thefour-bit nibble, in the MSG ADRS (23:0) message address signals, intowhich the HDR (3:0) header signals will be inserted. This will dependupon the number of flits in the down path identification portion 41 ofthe message, which, in turn, depends on the level "i" to which the datarouter node groups 20(i,j) transmit the message packet 30 in the datarouter 15. Essentially, the header nibble select logic circuit 372identifies the first nibble, beginning with the most-significant nibble,in the PHYS ADRS/DATA OUT (19:0) signals from the physical addresscomputation stage 324 (FIG. 9B-3) which do not all represent the valuezero. The header is inserted into the message address signals just abovethe identified nibble.

The HDR (3:0) header signals and the signals from the header nibbleselect logic circuit 372 are coupled to a message address assembly logiccircuit 373 (FIG. 9B-7). This circuit also receives the PHYS ADRS/DATAOUT (19:0) physical address/data out signals from the physical addresscomputation stage 324 (FIG. 9B-3). The message address assembly logiccircuit 373 generates the MSG ADRS (23:0) message address signals byinserting the HDR (3:0) signals into the four-bit nibble identified bythe signals from the header nibble select logic circuit 372. It will beappreciated that the message address assembly logic circuit 373essentially transmits the nibbles of the PHYS ADRS/DATA OUT (19:0)signals which do not represent the value zero as low-order MSG ADRSmessage address signals, and the HDR (3:0) header signals in the nexthigher-order nibble.

FIG. 9B-5 depicts the logic diagram of the header nibble calculationlogic circuit 371. With reference to FIG. 9B-5, the circuit 371 includesa height identifier circuit 374 that generates HGT (10:1) heightidentifier signals in response to the REL ADRS (19:2) relative addresssignals, and a decoder 375. The height identifier circuit 374 includesnine OR gates 376(2) through 376(10) [generally identified by referencenumeral 376(i)], each of which receives two of the REL ADRS (19:2)signals and generates a HGT (i) signal in response. If a HGT (i) signalis asserted, the message packet 30 will go to at least that level "i" toreach a data router node group 20(i,j) that is the root of the smallestsub-tree including both the source leaf 21(x) and the destination leaf21(y). If a HGT (i') signal having a higher index "i" is asserted, themessage packet 30 will have to go at least to that level "i'."

Thus, OR gate 376(10) receives the REL ADRS (19) and REL ADRS (18)signals and generates the HGT (10) height signal in response thereto. Ifeither the REL ADRS (19) or REL ADRS (18) signals are asserted, the rootof the smallest sub-tree of data router 15 including both the sourceleaf 21(x) and destination leaf 21(y) is at level 10. On the other hand,if, for example, the REL ADRS (19:14) signals are negated, and the RELADRS (13) signal is asserted, the displacement between the source leaf21(x) and the destination leaf 21(y) is not as great, and so the root ofthe sub-tree including both leaves will be at a lower level. In thatcase, the root of the smallest sub-tree including both the source anddestination leaves 21(x) and 21(y) is at level 7. OR gate 376(7), whichreceives the REL ADRS (13) signal, asserts the HGT (7) height signal inresponse to the asserted REL ADRS (13) signal.

Since the message packet 30 will always be transmitted to at least level1 in the data router 15, a HGT (1) height signal is maintained at anasserted level.

It will be appreciated that, depending on the particular encoding of theREL ADRS (19:2) relative address signals, several of the HGT (10:2)signals may be asserted. In any case, the HGT (i) signal that identifiesthe level "i" of the smallest sub-tree including both the source anddestination leaves 21(x) and 21(y) will always be asserted. The decoder375 generates HDR (3:0) header signals that are binary-encoded with thelargest index value "i" of the asserted HGT (i) signals. Thus, forexample, if the HGT (10) height signal is asserted, index (10) isrepresented by binary encoding (1010). To accommodate that encoding, theHGT (10) height signal enables an OR gate 377 to assert the HDR (3)signal, which provides the high-order "1" in the binary encoding (1010).The high-order "1" is also asserted in response to the assertions of theHGT (9) signal and the HGT (8) signal, and so the OR gate 377 is alsoconnected to OR gates 376(9) and 376(8).

Since the highest level in the embodiment of data router 15 disclosedherein is 10, if the high-order HDR (3) header signal is asserted, thesecond-order HDR (2) signal is never asserted; otherwise the encoding ofthe HDR (3:0) signals would represent binary-encoded values greater thanten. Accordingly, if the HDR (3) header signal is asserted, an inverter380 disables an AND gate 381 to maintain the HDR (2) signal at a negatedlevel. Thus, the HDR (2) signal would be negated even if a HGT (i)signal is asserted that would otherwise result in assertion of the HDR(2) signal.

Continuing with the example in which the HGT (10) signal is asserted,the asserted HGT (10) signal also enables an OR gate 382 to assert theHDR (1) signal, providing the third-order "1" in the binary encoding(1010). In addition, an inverter 383 disables an AND gate 384 to negatethe HDR (0) signal and providing the low-order "0." The inverter 383maintains the HDR (0) signal at a negated state even if other HGT (i)signals are asserted that would otherwise result in assertion of the HDR(1) signal.

The decoder 375 includes a number of other gates that selectively enableand disable gates 377, 381, 382 and 384 to enable generation of theappropriate HDR (3:0) header signals in response to the HGT (i) heightsignals. Since the operation of the decoder will be apparent to thoseskilled in the art, it will not be further described herein.

The header nibble select logic circuit 372 (FIG. 9B-4) is depicted inFIG. 9B-6. With reference to FIG. 9B-6, the circuit 372 comprises aheight-pair identifier circuit 385 and a decoder 386. The height-pairidentifier circuit includes a set of OR gates 387(5) through 387(2)[generally identified by reference numeral 387(i)], each of whichreceives four of the REL ADRS (19:4) relative address signals. Withreference to FIG. 9B-5 as well, it will be appreciated that each OR gate387(i) corresponds to two OR gates 376(i); that is, each OR gate 387(i)is energized by the REL ADRS signals when either of two OR gates 376(i)is energized thereby. Thus, for example, if either of OR gate 376(3) or376(4) is energized to assert the HGT(4) OR HGT(3) signals, indicatingthat the message packet 30 will be transmitted to at least either ofthose levels in the data router 15, OR gate 387(2) will also beenergized. Similarly, OR gate 387(3) is energized when either the HGT(5) or HGT (6) signal is asserted, OR gate 387(4) is energized when theHGT (7) or HGT (8) signal is asserted, and OR gate 387(5) is energizedwhen the HGT (9) or HGT (10) signal is asserted.

If an OR gate 387(i) is energized, the header 40 comprises flit "i" ofthe message address portion 31 of the message packet 30, counting fromthe first flit of the message address portion 31 which is identified asflit zero. If one of the REL ADRS (19:16) relative address signals areasserted, there are nine or ten down-path identifiers 42 requiring fiveflits in the down path identifier portion 41. Thus, if any of the RELADRS (19:16) signals is asserted, OR gate 387(5) asserts a HDR FLIT 5signal, indicating that the header 40 comprises flit "5" of the messageaddress portion 31. OR gates 387(4) through 387(2) operate similarly inresponse to the other REL ADRS (15:4) signals.

The decoder 386 determines the energized OR gate 387(i) with the highestindex value (i) and in response generates signals for controlling themessage address assembly logic circuit 373 (FIG. 9B-7). If the OR gate387(5) is energized, the decoder 386 asserts a HDR FLIT 5 signal,indicating that the header 40 is in flit "5" of the message addressportion 31. The decoder 386 includes three AND gates 390(2) through390(4), each of which has one input terminal connected to the output ofa respective OR gate 387(2) through 387(4). If the OR gate 387(4) isenergized, one input terminal of AND gate 390(4) is enabled. If the ORgate 387(5) is not energized, an inverter 391 asserts a HDR NOT FLIT 5signals, energizing the AND gate to assert a HDR FLIT 4 signalindicating that the header 40 comprises flit "4" of the message addressportion, and that there are four flits in the down path identificationportion.

Similarly, if OR gate 387(3) is energized, one input terminal of ANDgate 390(3) is enabled. If both the OR gates 387(4) and 387(5) aredisabled, a NOR gate 392 energizes the second input terminal of AND gate390(3), enabling it to assert the HDR FLIT 3 signal, indicating that theheader 40 comprises flit "3" of the message address portion 31, and thatthere are three flits in the down path identification portion 41. Aninverter 393 inverts the output of NOR gate 392 to assert an HDR FLIT4/5 signal, indicating that the header is in either flit "4" or "5". ANDgate 390(2) operates similarly in response to the energization of the ORgate 387(2) and a NOR gate 394. If the OR gate 387(2) is energized andOR gates 387(3) through 387(5) disabled, AND gate 390(2) asserts a HDRFLIT 2 signal, indicating that the header 40 comprises flit "2" of themessage address portion 31. If the NOR gate 394 is not energized, aninverter 395 asserts a HDR FLIT 3/4/5 signal indicating that the headercomprises either flit "3," "4," or "5."

Finally, if all of the OR gates 387(2) through 387(5) are disabled, aset of inverters 396 enables an AND gate 397 to assert a HDR FLIT 1signal, indicating that the header 40 is in flit "1" of the messageaddress portion 31, and that the down path identification portion 41 ofthe message packet includes one flit, namely, flit "0." If the AND gate397 is disabled, such that the HDR FLIT 1 signal is negated, an inverter398 asserts a HDR FLIT 2/3/4/5 signal indicating that the header 40 isin either flit "2," "3," "4," or "5."

The message address assembly logic 373 (FIG. 9B-7) uses the signals fromthe header nibble select logic 372 to control the insertion of the HDR(3:0) header signals in the MSG ADRS (23:0) message address signals.With reference to FIG. 9B-7, the message address assembly logic 373receives the PHYS ADRS/DATA OUT (19:0) physical address/data out signalsand directs four-bit nibbles to gate circuits 400(1) through 400(4)[generally identified by reference numeral 401(i)] which control theselection of the nibble of the MSG ADRS message address signals ontowhich the HDR (3:0) header signals are transmitted. Since the messageaddress portion 31 of a data router message packet 30 always includes atleast one flit in the down path identification portion 41, which isrepresented by the low-order nibble MSG ADRS (3:0) signals, the PHYSADRS/DATA OUT (3:0) signals are transmitted directly as MSG ADRS (3:0)message address signals.

Each gate circuit 400(1) through 400(4) includes a multiplexer 401(1)through 401(4) [generally identified by reference numeral 401(i)] and anAND gate 402(1) through 402(4) [generally identified by referencenumeral 402(i)]. In each gate circuit 400(i), the multiplexer 401(i)couples either the HDR (3:0) signals or a nibble of the PHYS ADRS/DATAOUT coupled to its associated AND gate 402(i) over one particular nibbleof the MSG ADRS message address signals, in response to the HDR FLIT (i)signal from the header nibble select logic 372. The respective AND gate402(i) is controlled by the HDR FLIT i+1/ . . . /5 signal from theheader nibble select logic.

Thus, if, for example, the header nibble select logic determines thatthe header comprises, flit "1" of the message address portion 372, theHDR FLIT 2/3/4/5 signal is negated, disabling the AND gate 402(1). TheHDR FLIT 1 signal is asserted, enabling the multiplexer 401(1) totransmit the HDR (3:0) header signals as the MSG ADRS (7:4) messageaddress signals. As noted above, the message address assembly logictransmits the PHYS ADRS/DATA OUT (3:0) physical address/data out signalsas the MSG ADRS (3:0) message address signals.

The other HDR FLIT (i) signals are negated, enabling the respectivemultiplexers 401(i) (i=2, 3 and 4) to couple the values at their "zero"data input terminals as the respective MSG ADRS signals. In that case,since the other HDR FLIT i+1/ . . . /5 signals are also negated, theother AND gates 402(i) are disabled, so that the input terminals tomultiplexers at the "zero" data input terminals are all zero, thusensuring that the respective MSG ADRS message address signalstransmitted thereby all represent the value zero. In addition, an ANDgate 403, which receives the HDR (3:0) signals, is disabled by thenegated HDR FLIT 5 signal, thereby ensuring that the MSG ADRS (23:20)message address signals all represent the value zero.

If, as another example, the header nibble select logic 372 is assertingthe HDR FLIT (3) signal, indicating that the header 40 comprises flit"3" of the message address portion 31, the HDR FLIT 2/3/4/5 and HDR FLIT3/4/5 signals are asserted, and the HDR FLIT 4, HDR FLIT 5 and HDR FLIT4/5 signals are negated. In that case, the multiplexer 401(3) is enabledto transmit the HDR (3:0) header signals as the MSG ADRS (15:12) messageaddress signals. Since the HDR FLIT 4, HDR FLIT 5 and HDR FLIT 4/5signals are negated, AND gates 402(3) through 402(5) are all disabled.As a result, the MSG ADRS (23:20) message address signals are allnegated, representing the value zero. Similarly, the negated HDR FLIT 4signal enables the multiplexer 402(4)to couple the signals from AND gate402(4) as the MSG ADRS (19:16) signal. Since the AND gate 402(4) isdisabled, the MSG ADRS (19:16) represent the value zero.

Continuing with the example, since the HDR FLIT 1 and HDR FLIT 2 signalsare also negated, multiplexers 401(1) and 401(2) transmit the signalsfrom their respective AND gates 402(1) and 402(2) as the respective MSGADRS signals. Since the AND gates are enabled by the asserted HDR FLIT2/3/4/5 and HDR FLIT 3/4/5 signals, the MSG ADRS (7:4) and MSG ADRS(11:8) signals, from multiplexers 401(1) and 401(2) correspond to thePHYS ADRS/DATA OUT (7:4) and (11:8) signals, respectively. The messageaddress assembly logic operates similarly if the header is to be inother flits in the message address portion 31.

Returning to FIG. 9B-4, as noted above, the staging register 366 latchesthe signals representing the message address, length, tag and data inresponse to the RUN signal from the transmitter stage 326 (FIG. 9B-1)and the NODE CLK signal, and contemporaneously the latch 368 latches theIS ADRS signal. The latches transmit the latched signals as the MSG OUT(32:0) message out signals, which are coupled to the transmitter stage326. FIG. 9B-8 depicts a detailed logic diagram of the transmitter stage326.

With reference to FIG. 9B-8, the transmitter stage includes a bufferregister 410 and a nibble counter 411. The buffer register latches theMSG OUT (32:0) message out signals and transmits the high order bit asan ADRS WD address word signal, and the remaining bits as XMIT OUT(31:0) transmitter out signals. If the ADRS WD address word signal isasserted, the XMIT OUT (31:0) signals correspond to a message address.

The nibble counter 411 transmits NC (7:0) nibble count signals each ofwhich is associated with one of the successive four-bit nibbles of theXMIT OUT (31:0) signals. When enabled by an asserted NC DEC EN nibblecounter decrement enable signal from an AND gate 417, the nibble counter411 decrements in synchrony with the NODE CLK node clock signal,iteratively asserting the NC 7 through NC 0 nibble count signals. TheAND gate 417 is energized while the FLY IN signal is asserted, if anormally-negated CHECK OUT signal is negated.

When the NC 0 nibble count signal is asserted, it enables the bufferregister 410 to latch the MSG OUT (32:0) message out signals from themessage address computation stage 325 (FIG. 9B-4) at the next tick ofthe NODE CLK signal. The NC 7 nibble count signal is transmitted to themessage address computation stage 325 and the physical addresscomputation stage 324. When the NC 7 nibble count signal is asserted,which occurs after the current contents of the staging register 366 andlatch 368 in the message address computation stage 325 have been latchedin the buffer register 410, the NC 7 nibble count signal enables thosestages to operate, to generate new MSG OUT (32:0) message out signals.

The NC (7:0) nibble count signals also control two circuits in thetransmitter stage 326. A flit selection circuit 413, in response to theNC (7:0) nibble count signals, selects successively lower-order four-bitnibbles of the XMIT OUT (31:0) transmitter out signals for transmissionas the four-bit FLIT IN signals. In addition, a start of message detectcircuit 414 scans the nibbles of the XMIT OUT (31:0) signals, and, ifthe XMIT OUT (31:0) signals represent an address word, as indicated byan asserted ADRS WD address word signal, asserts an SOM start of messagesignal.

The transmitter control circuit 412 uses the SOM start of message signalto identify the first flit of a data router message packet 30. When theSOM signal is asserted, if the FLY IN signal is asserted it asserts anEN OUT enable out signal, which enables the nibble selection circuit 413to begin transmitting nibbles in synchrony with ticks of the NODE CLKsignal. A normally-negated CHECK OUT signal from the transmitter control412 enables a multiplexer 415 to couple the output from the nibbleselection circuit 413 as FLIT IN signals.

In addition, the transmitter control circuit 412 asserts a HEADER signalthat resets and enables a check flit generator 416. The check flitgenerator 416 receives LAT FLIT (3:0) signals from the nibble selectioncircuit 413, that, at each tick of the NODE CLK signal, arerepresentative of the flit then being transmitted, and generates errordetection check bits in response thereto.

The transmitter control circuit 412 also receives the LAT FLIT (3:0)latched flit signals representative of the header 40 of the messagepacket 30 and determines therefrom the number of flits in the messageaddress portion 31. After that number of flits have been transmitted,the next flit corresponds to the data router message packet's messagelength field 34. The transmitter control circuit 412 uses the valueidentified by that flit in determining when the message data portion 32has been transmitted. When the transmitter control circuit 412determines that the message data portion has been transmitted, itasserts the CHECK OUT signal, which enables the multiplexer 415 tocouple CHECK FLIT (3:0) signals from the check flit generator as theFLIT IN signals. The asserted CHECK OUT signal also disables the nibblecounter 411, inhibiting it from decrementing.

The start of message detection circuit 414 includes a plurality of ORgates 420(3) through 420(7) [generally identified by reference numeral420(i)]. Each OR gate 420(i) receives the signals from the "i-th" nibbleof the XMIT OUT (31:0) transmitter output signals. If all of the signalsof the nibble coupled to OR gate 420(i) are negated, the OR gateproduces a negated output signal. On the other hand, if one of thesignals of the nibble is asserted, the OR gate produces an assertedoutput signal.

The output of each OR gate 420(i) is connected to the input terminal ofone of AND gates 421(3) through 421(7). The other input terminal of eachAND gate 423 (i) is connected to receive a corresponding one of the NC inibble counter signals. Thus, the nibble counter 411 decrements,successively asserting the NC 7 through NC 3 nibble count signals,enabling the successive AND gates 421(7) through 421(3). If an OR gate420(i) is energized by the assertion of at least one of the signals inthe nibble coupled thereto, the associated AND gate 421(i) is energizedto assert a corresponding one of the SOM "i" start of message (nibble"i") signals ("i" is an integer from seven to three, corresponding tothe nibble). When the first SOM "i" signal is asserted, an OR gate 422is energized to assert an SOM DET start of message detect signal.

It will be appreciated that the SOM DET start of message detect signalis asserted whenever a nibble of the XMIT OUT (31:12) transmitter outsignals includes an asserted signal, in synchrony with the correspondingNC "i" nibble count signal. The SOM DET start of message detect signalis coupled to an input terminal of an AND gate 423. If the AND gate isenabled by asserted ADRS WD address word and asserted FLY IN signals,which occurs if the buffer register 410 contains the first word of a newdata router message packet 30 and the transmitter stage 326 is enabledto transmit, the AND gate 423 asserts the SOM start of message signal.

It will be appreciated that the start of message detection circuit 414of the embodiment disclosed herein does not need elements correspondingto OR gates 420(i) or AND gates 421(i) for "i" equal to zero, one ortwo. The associated nibbles, comprising XMIT OUT (11:0) transmitter outsignals, if they are in a word which contains the first flit of a newdata router message packet, would contain the message length and messagetag fields 34 and 35 and first flit of the down path identificationportion 41. Thus, the nibble containing the header, which must containat least one asserted signal, must be in the XMIT OUT (31:12) signals.

The nibble selection circuit 413 receives the nibbles of the XMIT OUT(31:0) signals and selectively transmits them under control of the ENOUT enable out signal from the transmitter control circuit 412. Thefour-bit nibbles comprising XMIT OUT (31:4) signals are coupled toassociated data input terminals of a selector 424. Under control of theNC (7:0) nibble count signals, the selector 424 gates the signals fromthe corresponding "i-th" nibble to the input terminal of a flit buffer426, which latches the selected signals in response to the next tick ofthe NODE CLK signal.

The low-order nibble, comprising the XMIT OUT (3:0) signals, is loadedinto a buffer 425. The buffer 425 is enabled in response to the NC 7nibble count signal, and it latches the nibble in response to the nexttick of the NODE CLK signal. It will be appreciated that the buffer 425latches the low-order nibble contemporaneous with the transmission bythe selector 424 of the high-order nibble, that is, the XMIT OUT (31:27)transmitter out signals. The selector 424 transmits the contents ofbuffer 425 to the input terminal of flit buffer 426 in response to theNC 0 nibble count signal. The latching of the low-order nibble in thebuffer 425 permits the contents of the buffer register 410 to be updatedin response to the NC 0 nibble count signal, making the next wordavailable for transmission immediately after transmission of thelow-order nibble in buffer 425.

As noted above, the output of the selector 424 are latched in the flitbuffer 426 in response to the NODE CLK signal. The output of flit buffer426, namely, the LAT FLIT (3:0) latched flit signals, are coupled to thecheck flit generator 416, the transmitter control circuit 412, and alsoto the input terminal of a gated driver 427. When the transmittercontrol circuit 412 asserts the EN OUT enable output signal, the gateddriver 417 couples the LAT FLIT (3:0) latched flit signals to one set ofdata input terminals of multiplexer 415. As described above, if thetransmitter control circuit 412 is negating the CHECK OUT signal, themultiplexer 415 couples the contents of flit buffer 426 as the FLIT INsignal to the data router node 22(i,j,k) connected thereto.

As described above, the data router interface 205 includes two messageinjector ports 223(l) and 223(r). The processor 200 (FIG. 8) controllingthe processing element 11 can select one of the message injector portsto inject the data router message packet 30 into the data router 14, inwhich case the information used in forming the message packet 30 arecoupled to the identified message injector port 223. On the other hand,the processor 200 can initiate injection of a data router message packet30 by referencing a "middle" message injector port, and the injectorcommon control/status portion 224 will select one of the left or rightports 223 to inject the message packet 30. FIG. 9B-9 depicts a targetselect circuit 426, in the injector common control/status portion 224,for accomplishing this.

With reference to 9B-9, the target select circuit receives (L) FIFO FULLand (R) FIFO FULL FIFO full status signals from the first-in first-outbuffers 321 (FIG. 9B-1) in the left and right injector ports 223(l) and223(r), respectively. The target select circuit 426 includes a set ofAND gates that use these status signals, and pointer signals CUR TARGETL current target left and CUR TARGET R current target right generated bya flip-flop 442 and inverter 443. The assertion or negation of the CURTARGET L and CUR TARGET R identify whether the left or right injectorport will be selected to transmit a data router message packet 30addressed by the processor 200 to the "middle" message injector port.

AND gate 430 is energized by the coincident assertion of the (L) FIFOFULL signal and of the CUR TARGET L signal to assert a CUR L FULLcurrent left/full signal. The assertion of the CUR L FULL signalindicates that the left injector port 223(l) is currently selected bythe target select circuit 426, but that its FIFO 321 is full. Similarly,AND gate 431 is energized by the coincident assertion of the (R) FIFOFULL signal and of the CUR TARGET (R) signal to assert a CUR R FULLcurrent right/full signal. The assertion of the CUR R FULL signalindicates that the right injector port 223(4) is currently selected bythe target select circuit, but that its FIFO 321 is full.

AND gate 432 is energized by the coincident assertion of the (L) FIFOFULL signal and the CUR TARGET R signal to assert a NEW R FULL L newright/full left signal. The assertion of the NEW R FULL L signalindicates that the right injector port 223(r) is currently selected bythe target selector circuit and that the FIFO 321 of the left injectorport 223(l) is full. Similarly, AND gate 433 is energized by thecoincident assertion of the (R) FIFO FULL signal and the CUR TARGET Lsignal to assert a NEW L FULL R new left/full right signal. Theassertion of the NEW L FULL R signal indicates that the left injectorport 223(l) is currently selected by the target selector circuit 426 andthat the FIFO 321 of the right injector port 223(r) is full. The NEW RFULL L and NEW L FULL R signals are coupled to input terminals of an ORgate 435, whose output is complemented by an inverter 436 to generate aNEW TARG NOT FULL new target not full signal.

The CUR L FULL, CUR R FULL and NEW TARG NOT FULL signals are coupled toan OR gate 437, which generates a TOGGLE EN toggle enable signal. If anOR gate 440 is energized, which occurs when the processor 200 enablesthe data router interface 205 to inject a new data router message packet30, it asserts a NEW MSG new message signal. If the TOGGLE EN and NEWMSG signals are asserted, and AND gate 441 is energized, which, in turn,enables the clock input terminal of flip-flop 442. The flip-flop 442generates the CUR TARGET L current target left signal, which is coupledto AND gates 430 and 433. In addition, the CUR TARGET L signal iscomplemented by inverter 443 to generate the CUR TARGET R current targetright signal is coupled to the AND gates 431 and 433. The CUR TARGET Rsignal is also coupled to the data input terminal of flip-flop 442, sothat, when the AND gate 441 is energized, the flip-flop 442 toggles itsstate. The CUR TARGET L and CUR TARGET R signals are also coupled tocircuitry (not shown) that controls loading of information into thewrite stages 320 of the respective left and right message injector ports223(l) and 223(r), thus selecting the respective ports in response tothe conditions of their respective buffers 321 and the port throughwhich the last data router message packet 30 was being injected.

While the message injector port 223 has been described as includingciruits all of which operate in response to the NODE CLK signal providedby the clock buffer 207, it will be appreciated that that will normallyrequire substantial portions of the network interface 202, and in somecases the entire processing element 11 (FIG. 8) to operate in responseto the NODE CLK signal. In many cases, it may be desirable to have mostof the processing element 11, including most of the network interface202, to operate in response to a processing element clock signal (notshown) which synchronizes most operations on the processing element 11,and only small portions of the network interface 202 operate in responseto the NODE CLK signal. In particular, it may be desirable to have atleast the stages of the message injector ports 310 through 325 (FIG.9B-1), that is, those stages in advance of the transmitter stage 326,operate in response to the processing element clock signal.

In that situation, the various circuits of the transmitter stage 326, asshown on FIG. 9B-8, will operate in response to the NODE CLK signal fromthe clock buffer 207. In addition, the RUN signal, generated by thenibble counter 411, will not be coupled directly to the message addresscomputation stage 325 and physical address computation stage 324.Instead, the RUN signal will be directed to a synchronizer 444 thatgenerates, in response to the RUN signal, a SYNCH RUN synchronized runsignal that controls the message address computation stage 325 andphysical address computation stage 324.

FIG. 9B-9 depicts details of the synchronizer 444. With reference toFIG. 9B-9, synchronizer 444 includes a set-reset flip-flop 445 that isset in response to the assertion of the RUN signal. Flip-flop 445includes direct set and direct reset terminals, identified as "S" and"R", respectively. When a signal applied to either terminal changescondition from negated to asserted, the flip-flop is, respectively, setor reset.

The set flip-flop 445 energizes the data input terminal of a secondflip-flop 446, which operates as a buffer. In response to the next tickof a PE CLK processing element clock signal, the flip-flop 446 is set,to assert a BUF RUN signal. The asserted BUF RUN signal, in turn,enables a state machine 447, which actually asserts the SYNCH RUNsynchronized run signal to control the message address computation stage325 and physical address computation stage 324.

The state machine 447, which is clocked in response to the PE CLKsignal, has a state diagram which is also shown on FIG. 9B-9. Inparticular, the state machine 447 has three states, namely, an initialstate, represented by the box labelled "INIT", an intermediate state,represented by the box labelled "MID," and a synchronized run assertstate, represented by the box labelled "SYNCH RUN," in which it assertsthe SYNCH RUN signal. Transitions between states occur in synchronismwith the PE CLK signal.

The state machine 447 is initially in the initial state. When the BUFRUN buffered run signal is asserted, the state machine sequences to theintermediate state at the next tick of the PE CLK signal. Regardless ofthe condition of the BUF RUN signal, the state machine 447 sequences tothe synchronized run assert state at the next tick of the PE CLK signal.As noted above, the state machine 447 asserts the SYNCH RUN signal whenin the synchronized run assert state. The assertion of the SYNCH RUNsignal, in addition to controlling the message address computation stage325 and physical address computation stage 324, also resets theflip-flop 445, enabling the flip-flop 446 to be reset at the next tickof the PE CLK signal, to negate the BUF RUN signal. The next tick of thePE CLK signal also sequences the state machine 447 to the initial state.Since at that point the BUF RUN signal is negated, the state machinewill remain in the initial state.

iii. Message Ejector Portion 221

With reference again to FIG. 9A-1, the data router interface 201includes left and right message ejector ports 225(l) and 225(r). Sincethe left and right message ejector ports are generally similar, only one(without reference to it being the left or right port) will bedescribed. FIG. 9C-1 depicts a general block diagram of a messageejector port 225, and FIGS. 9C-2 through 9C-7 depict more detailed logicdiagrams of selected elements depicted in FIG. 9C-1.

With reference to FIG. 9C-1, the message ejector port includes a flitreceiver stage 450, an ejector flit first-in first-out buffer (FIFO)451, a message assembler 452 and a bus interface 453. The flit receiverstage is connected to the data router 15 and receives the successivefour-bit OUT FLIT signals from the data router node 22(1,j,k) connectedthereto. The flit receiver stage also performs an error detectionoperation, in which it verifies correct transmission of the data routermessage packet through the data router 15. The flit receiver stage 450controls loading of nibbles representative of the successively-receivedflits into the ejector first-in first-out FIFO 451.

The message assembler 452 receives the successive four-bit nibbles fromthe FIFO 451 and assembles thirty-two bit words in response thereto. Thethirty-two bit words are available to the processor 200 through therespective receive register 260, 295 or 302. Each of the successivethirty-two bit words receive in a message packet effectively has thesame word organization as the successive words transmitted by the sourceleaf 21(x).

The bus interface 453 controls the transmission of data from data routermessage packets 30 to the processor. The bus interface 453 also has afirst-in first-out buffer to buffer data received from the data router15 before it can be retrieved by the processor 200.

FIG. 9C-2 depicts a detailed logic of the flit receiver stage 450. Withreference to FIG. 9C-2, the flit receiver stage includes an AND gate 460that is enabled by a normally asserted FLOW EN flow enable signal from areceiver stage control circuit 461. If the FLOW EN flow enable signal isasserted, and if a FIFO ALMOST FULL signal from FIFO 451 is notasserted, the AND gate 461 asserts an OUT FLY signal that is transmittedto the data router node 22(i,j,k) connected thereto. The FLOW EN flowenable signal may be conditioned by a register (not shown) that may beconditioned by the diagnostic network 16 to effectively enable ordisable the ejector port 225. The FIFO 451 asserts the FIFO ALMOST FULLsignal when it is nearly full to regulate flow of flit data thereto forstorage.

The output terminal of AND gate 460 is also coupled to a delay line 461so that, when the OUT FLY signal is asserted, a MSG FLOW message flowsignal is asserted a brief time thereafter. The MSG FLOW signal iscoupled to a receiver stage control circuit 462.

The four-bit OUT FLIT (3:0) output flit signals from the connected datarouter node 22(1,j,k) are received at input terminals of a latch 463 andlatched in response to successive ticks of the NODE CLK signal. Thecontents of latch 463 are coupled as LAT FLIT (3:0) latched flit signalsto input terminals of the receiver stage control circuit 462, to amessage check circuit 464, and to one set of input terminals of amultiplexer 465.

The receiver stage control circuit 462 receives the LAT FLIT (3:0)latched flit signals from the latch 463 in synchrony with the NODE CLKsignal. If the immediately-preceding data router message packet 30 hasbeen completely received, and if a SOM COND DET start-of-message signalfrom an OR gate 466 is asserted, the receiver stage control circuit 462asserts a HEADER signal. The OR gate 466 generates the SOM COND DETstart-of-message condition detect signal if at least one of the four OUTFLIT (3:0) output flit signals is asserted, indicating the start of anew data router message packet 30 if the immediately-preceding datarouter message packet has been completely received.

The asserted HEADER signal resets the message check circuit 464 andenables it to initiate a check operation in connection with the LAT FLIT(3:0) latched flit signals in synchrony with the NODE CLK signals, whileit is enabled the MSG FLOW message flow signal is asserted. If the MSGFLOW signal is negated, the message check circuit 464 is disabled. Theoutput of the message check circuit 464 is coupled to a second set ofdata input terminals of multiplexer 465.

The multiplexer 465 is controlled by a CHECK IN signal from the receiverstage control circuit 462. The CHECK IN signal, when negated, enablesthe multiplexer 465 to couple the LAT FLIT (3:0) latched flit signals asRCV DATA (3:0) received data signals to the input terminal of ejectorfirst-in first-out FIFO 451 (FIG. 9C-1). The HEADER signal from receiverstage control circuit 462 is also coupled to the input terminal of FIFO451 as a high-order RCV DATA (4) signal. The HEADER signal, since it isasserted during the receipt of the first flit of a data router messagepacket 30 and negated otherwise, is used as a start of message indicatorin successive stages of the message ejector port 225.

The receiver stage control circuit 462, while the MSG FLOW signal isasserted, asserts a WRITE FIFO signal that enables the FIFO 451 to latchthe successive five-bit RCV DATA (4:0) receive data words, representingsuccessively-received flits, in synchrony with the NODE CLK signal. Itwill be appreciated that, if the FIFO 451 asserts the FIFO ALMOST FULLsignal, which negates the MSG FLOW signal, the receiver stage controlcircuit 462 will negate the WRITE FIFO signal, disabling loading ofadditional data into FIFO 451.

As indicated above, the receiver stage control circuit 462 also receivesthe LAT FLIT (3:0) latched flit signals. The receiver stage controlcircuit specifically latches the signals of the header field 40 and themessage length 34 of the data router message packet 30 and uses them toindicate when the receiver stage 30 has received all flits of a datarouter message packet 30. When the receiver stage control circuitdetermines that all of the flits have been received, it asserts theCHECK IN signal, enabling the multiplexer 465 to couple CHECK VERIFYsignals from the message check circuit 464, which indicate whether thedata router message packet was correctly transferred through the datarouter 15, as the RCV DATA (3:0) receive data signals. The receiverstage control circuit 462 may then, in response to the next assertion ofthe SOM COND DET start-of-message condition detect signal by the OR gate466, determine that the OUT FLIT (3:0) signals represent the first flitof a new data router message packet, in which case it enables theoperations as described above to be repeated.

The first-in first-out FIFO 451 buffers the RCV DATA (4:0) receive datasignals, representing five-bit words, from the flit receiver stage 450.In response to READ FIFO signals from the ejector message assembler 452,the FIFO 451 transmits the buffered words to the message assembler 452as EJ FIFO OUT (4:0) ejector first-in first-out buffer out signals.

The message assembler 452 receives successive words represented by thefive-bit EJ FIFO OUT (4:0) signals and forms, from the low-order fourbits, thirty-two bit words. The message assembler 452 assembles thefirst thirty-two bit word from the contents of the header, messagelength and message tag fields 40, 34 and 35 of a received data routermessage packet 30. In addition, if the data router message packet 30 wasreceived while the data router 15 is in all-fall-down mode, the firstword includes the contents of the down path identification portion 41,as received by the ejector port 225. The message assembler 452 formseach of the succeeding thirty-two bit words from the successive dataflits 36. Each of the successive thirty-two bit words formed by themessage assembler 452 from the data flits 36 from a data router messagepacket 30 received by an ejector port 225 at a destination leaf 21(y)have the same contents as the successive thirty-two bit words from whichdata router message packet 30 was formed by the injector port 223 at thesource leaf 21(x). Accordingly, the binary-encoded values represented bythe successive thirty-two bit words received by the ejector port 225 atthe destination leaf 21(y) are the same as the binary-encoded valuesrepresented by the thirty-two bit words transmitted by the injector port223 at the source leaf 21(x).

In the message assembler 452, a header delay circuit 470 generates anormally-asserted ASSY CTRL EN assembly control enable signal, whichenables one input terminal of an AND gate 472. If a STALL signal fromthe bus interface 453 is not asserted, an inverter 471 asserts a RUNsignal, which energizes an AND gate 472 to assert a CTRL EN controlenable signal. If the first-in first-out FIFO 451 is not empty, itnegates a FIFO EMPTY signal, which is complemented by an inverter 473 toenable an AND gate 474. The asserted CTRL EN control enable signalenergizes the AND gate 474 to assert a READ FIFO signal. While the READFIFO signal is asserted, the FIFO 451 transmits successive five-bitwords, defined by the EJ FIFO OUT (4:0) signals, in synchrony with theNODE CLK signal.

The bus interface 453 maintains the STALL signal in a negated conditionif it can receive data from the message assembler 452. When the businterface 453 cannot receive data from the message assembler 452, itasserts the STALL signal to negate the RUN signal. The negated RUNsignal disables the AND gate 472 to negate the CTRL EN signal, therebyenabling the AND gate 474 to negate the READ FIFO signal. In addition,if the FIFO 451 asserts the FIFO EMPTY signal, inverter 473 disables theAND gate 474, enabling it to negate the READ FIFO signal. While the READFIFO signal is negated, transmissions from the FIFO 451 stop.

A multiplexer 475 has one data input terminal connected to receive thefive-bit word defined by the EJ FIFO OUT (4:0) signals from the first-infirst-out FIFO 451. If the FIFO EMPTY signal is asserted, themultiplexer 475 couples negated signals, having a binary-encoded valueof zero, as SEL DATA selected data signals to the input terminal of aflit buffer 476, which latches them in response to successive ticks ofthe NODE CLK signal.

On the other hand, if the FIFO EMPTY signal is negated, the multiplexer475 couples the EJ FIFO OUT (4:0) signals as SEL DATA selected datasignals to the input terminal of a flit buffer 476 and the high-order EJFIFO OUT (4) signal as an SEL DATA (4) (HEADER) signal to an assemblycontrol circuit 477. The SEL DATA (4) (HEADER) signal indicates whetherthe remaining EJ FIFO OUT (3:0) signals comprise the header of thereceived data router message packet 30. The assembly control circuit477, in response to the CTRL EN control enable signal from AND gate 472,and in response to successive ticks of the NODE CLK signal, generatestiming control signals for controlling the other elements of the messageassembler 452.

Each tick of the NODE CLK signal enables the buffer 476 to latch SELDATA selected data signals defining a five-bit word as received from theFIFO 451. The buffer 476 transmits the latched signals as BUF SEL DATA(4:0) buffered selected data signals. If the BUF SEL DATA (3:0) signalsfrom flit buffer 476 represent the header field 40 of the data routermessage packet 30, the high-order BUF SEL DATA (4) buffered selecteddata signal from buffer 476 is asserted, which enables an AND gate 481.If the RUN signal is also asserted, the AND gate 481 is energized toenable a load enable input terminal of the header delay circuit 470. Theheader delay circuit 470 receives BUF SEL DATA (3:0) signals in responseto the next tick of the NODE CLK signal. It will be appreciated that atthat point the BUF SEL DATA (3:0) signals buffered by flit buffer 476represent the header field 40 of the message packet 30. At that point,the header delay circuit 470 negates the ASSY CTRL EN assembly controlenable signal, which, as described above, results in negation of theREAD FIFO signal.

The header delay circuit 470 maintains the ASSY CTRL EN assembly controlenable signal negated for a number of ticks of the NODE CLK signal whichdepends on the binary-decoded value of the BUF SEL DATA (3:0) signals,to provide timing synchronization with the bus interface 453. The numberof NODE CLK ticks that the ASSY CTRL EN signal remains negated is, inturn, related to the number of flits in the down path identificationportion 41 of the received data router message packet 30, which, inturn, relates to the binary encoded value of the BUF SEL DATA (3:0)signals.

The assembly control circuit 477 generates timing and control signalsthat control the arrangement of four-bit nibbles, represented by thesuccessive low-order BUF SEL DATA (3:0) buffered selected data signalsinto thirty-two bit words, which are assembled in an assemblysynchronizer register 480.

The assembly synchronizing register 480 includes eight four-bit nibblesidentified by reference numerals 480(7) through 480 (0) [generallyidentified by reference numeral 480(i)], a beginning-of-message flag 486and an all-fall-down mode flag 487. The beginning-of-message flag 486,when set, indicates that the word assembled in nibbles 480(i) comprisethe first word from a data router message packet 30 being received. Theall-fall-down mode flag 487, when set, indicates that the data routermessage packet 30 is being received while the data router 15 is inall-fall-down mode. The all-fall-down mode flag 487 is used to conditionthe received all-fall-down flag 254 of the appropriate private register232, 294 or 301 (FIGS. 9A-2A and 9A-2B).

In addition, the assembly control circuit 477 generates a VALID WORDsignal, which is transmitted to the bus interface 453. If asserted, theVALID WORD signal indicates that the message assembler 452 has assembleda thirty-two bit word at register 480 for transmission to the businterface 453. The bus interface 453 controls the latching of thethirty-two bit word by the register 480.

If the SEL DATA (4) (HEADER) signal from multiplexer 475 is asserted,the assembly control circuit 477 receives the latched BUF SEL DATA (3:0)signals defining the header from the buffer 476. The assembly controlcircuit 477 uses the BUF SEL DATA (3:0) signals to determine whether thedata router message packet 30 is being received while the data router 15all-fall-down mode. If the data router 15 is not in all-fall-down mode,the data router node (1,j,k) connected to the ejector port 225 will havedecremented the contents of the header field 40 of the data routermessage packet 30 to a condition in which all of the BUF SEL DATA (3:0)buffered selected data signals are asserted, representing thebinary-encoded value "fifteen." In addition, the data router node(1,j,k) will have discarded the last flit of the down pathidentification portion 41, and so only the message data portion 32remains after the header field 40.

As a result, the assembly control circuit 477 generates timing andcontrol signals that enable the BUF SEL DATA (3:0) signals representingthe header, length and tag fields 40, 34 and 35 to be latched as thefirst word in the assembly synchronizing register 480. Thereafter, thetiming and control signals from the assembly control circuit 477 enablethe assembly synchronizing register 480 to form successive thirty-twobit words from successive sets of eight four-bit words, each of thefour-bit words being defined by the BUF SEL DATA (3:0) signals atsuccessive ticks of the NODE CLK signal. The assembly control circuit477 enables the assembly synchronizing register to form a number ofthirty-two bit words, the number corresponding to the value in thelength field 34. Thereafter, the assembly control circuit enables theassembly synchronizing register to form another word to accommodate BUFSEL DATA (3:0) signals comprising the check bits as generated by themessage check circuit 464 (FIG. 9C-2).

On the other hand, if the assembly control circuit 477 determines fromthe BUF SEL DATA (3:0) signals that the data router message packet 30being received while the data router 15 is in all-fall-down mode, theBUF SEL DATA (3:0) buffered selected data signals that define the headerfield 40 are not all asserted. As noted above, when in all-fall-downmode, the data router nodes 22(i,j,k) in the data router 15 willmaintain the contents of the header fields 40 of the data router messagepackets 30 at their respective values at the time the all-fall-down modeis initiated. In one particular embodiment, the number of levels in thedata router 15 is selected so that the binary-encoded value of the BUFSEL DATA (3:0) signals will be less than "fifteen." Thus, the assemblycontrol circuit 477 can determine whether the data router 15 is in anall-fall-down mode by determining whether the binary-encoded valuerepresented by the BUF SEL DATA (3:0) signals is "fifteen."

As also noted above, when in all-fall-down mode, data router messagepackets that the ejector port 225 receives include at least some flitscomprising the down path identification portion 41. In that case, theassembly control circuit 477 generates timing and control signals thatenable the BUF SEL DATA (3:0) signals representing the down pathidentification portion 41, as well as those representing the header,length and tag fields 40, 34 and 35, to be latched as the first word inthe assembly synchronizing register 480. Thereafter, the timing andcontrol signals from the assembly control circuit 477 enable theassembly synchronizing register 480 to form successive thirty-two bitwords in the same manner as when the data router 15 is not inall-fall-down mode.

With this background, the assembly control circuit 477 includescircuitry, which is described below in connection with FIGS. 9C-4 and9C-5 that successively generates FLIT "i" OF WORD signals ("i" is aninteger from seven to zero) that control coupling of the low-order BUFSEL DATA (3:0) signals for latching in particular nibbles of theassembly synchronizing register 480. The BUF SEL DATA (3:0) bufferedselected data signals are coupled to the input terminals of respectivegates 481(7) through 481(0) [generally identified by reference numeral481(i)] which are controlled by the respective FLIT "i" OF WORD signals.The output terminals of the gates 481(i) are connected to respectiveones of buffers 482(7) through 482(0) [generally identified by referencenumeral 482(i)] which latch and buffer the signals gated by therespective gates 481(i). Effectively, each buffer 482(i) latches thesignals gated thereto by the respective buffer 481(i), in response to anenabling signal comprising the coincidence of the FLIT "i" OF WORDSIGNAL and the NODE CLK signal, the enabling signal being delayed toaccommodate delay of propagation of the gated signals to the inputterminals of the respective buffer 481(i).

The output terminals respective buffers 482(7) through 482(0) areconnected to input terminals of respective gates 485(7) through 485(0)[generally identified by reference numeral 485(i)]. In response to anasserted LAT WORD latch word signal from the assembly control circuit477, the gates 485(7) and 485(5) through 485(1) couple the signalsdirected thereto by the buffers 482(i) to respective nibbles 480(i) ofthe assembly synchronizing register 480. The nibbles 480(i) all latchthe signals in unison in response to the assertion of the WRITE ASRwrite assembly synchronizing register signal from the bus interface 453.

The input signals to nibbles 480(0) and 480(6) of the assemblysynchronizing register 480 are provided by multiplexers 483 and 484,respectively. The multiplexer 483, under control of a DATA NIB (0) datanibble "zero" signal from an AND gate 486, selectively couples theoutput signals from either gate 481(0) or 485(0) to the nibble (0) ofthe assembly synchronizing register 480. Similarly, the multiplexer 484,under control of a TAG signal from the assembly control circuit 477,selectively couples the output signals from either the gate 481(6) or485(6) to the nibble (6) of the assembly synchronizing register 480.

The assembly control circuit 477 controls the assembly synchronizingregister 480, the gates 481(i), buffers 482(i) and gates 485(i), as wellas multiplexers 483 and 484, as follows. When the SEL DATA (4) (HEADER)signal is asserted, the assembly control circuit 477 latches the BUF SELDATA (3:0) buffered selected data signals comprising the header field 40of the data router message packet 30 being received. The assemblycontrol circuit 477 asserts the FLIT 7 OF WORD signal, enabling the gate481(7) to couple the BUF SEL DATA (3:0) signals to the input terminal ofbuffer 482(7), which latches them in response to the next NODE CLKsignal.

As noted above, the BUF SEL DATA (3:0) signals correspond to thecontents of the message packet's header field 40. If the signalsindicate that the data router 15 is in all-fall-down mode, the assemblycontrol circuit 477 asserts a DNF LOOP down flit loop signal and enablesa down flit counter, described below in connection with FIG. 9C-5. Thedown flit counter generates count signals that control generation of theFLIT "i" OF WORD signals to enable gating and buffering of BUF SEL DATA(3:0) signals representing the successive flits of the down pathidentification portion 41 in the successive ones of buffers 482(i). Inone particular embodiment, in which data router message packets 30 havea maximum of five flits in the down path identification portion, theFLIT "i" OF WORD signals enable the BUF SEL DATA (3:0) signals to besuccessively buffered in buffers 482(4) through 482(0). If a particularmessage packet 30 has fewer than five flits in the down pathidentification portion, the FLIT "i" OF WORD signals enable the bufferedBUF SEL DATA (3:0) signals to be packed toward the buffers 482(i) withlower indices "i"; that is, if a data router message packet 30 has only"j" flits ("j" less than five) in the down path identification portion41, the assembly control circuit 477 successively generates the FLIT"j-1" OF WORD through FLIT 0 OF WORD signals, to enable the BUF SEL DATA(3:0) signals representative of those flits to be loaded into buffers482(j-1) through 482(0).

In addition, when the assembly control circuit 477 asserts the FLIT 0 OFWORD signal, a multiplexer 490 couples the DNF LOOP signal, which setsthe all-fall-down mode flag 487.

After the BUF SEL DATA (3:0) signals representative of the down pathidentification portion 41 have been buffered in appropriate ones ofbuffers 482(4) through 482(0), the assembly control circuit successivelyasserts a LEN length signal and a TAG signal, in synchronism withsuccessive ticks of the NODE CLK signal. Contemporaneous with itsassertion of the LEN length signal, the assembly control circuit 477also asserts the FLIT 5 OF WORD signal, enabling the BUF SEL DATA (3:0)signals to be gated to and latched by buffer 482(5). At that point theBUF SEL DATA (3:0) buffered selected data signals are representative ofthe length field 34 of the data router message packet 30. In addition,the assembly control circuit 477 stores the length information from theBUF SEL DATA (3:0) signals for its later use in assembling thirty-twobit words from data flits 36.

Contemporaneous with its assertion of the TAG signal, the assemblycontrol circuit 477 asserts the FLIT 6 OF WORD signal, enabling gate481(6) to couple the BUF SEL DATA (3:0) buffered selected data signalsto the input terminal of both buffer 482(6) and to one set of data inputterminals of multiplexer 484. Since the TAG signal is asserted, themultiplexer 484 couples the output of gate 481(6) directly to the inputterminal of the respective nibble (6) of the assembly synchronizingregister 480. In addition, the TAG signal is coupled to one inputterminal of a multiplexer 491, which controls the input tobeginning-of-message flag 486 of the assembly synchronizing register480.

Contemporaneously with the assertion of the TAG signal, the assemblycontrol circuit 477 asserts the LAT WORD latch word signal, whichenables the gates 485(7) and 485(5) through 485(1) to couple signalsfrom buffers 482(7) and 482(5) through 482(1) to respective nibbles (7)and (5) through (1) of the assembly synchronizing register. In addition,the LAT WORD signal enables the gate 485(0) to couple signals frombuffer 482(0) to the multiplexer 483. Since the DATA NIB (0) data nibblesignal is negated, multiplexer 483 couples the signals from gate 485(0)to nibble (0) of the assembly synchronizing register 480. In addition,the assembly control circuit asserts the VALID WORD signal, to indicateto the bus interface that a word is available at the input terminals ofthe assembly synchronizing register 480.

When the bus interface 453 can receive the word from the messageassembler 452, it asserts the WRITE ASR write assembly synchronizingregister signal, which enables the assembly synchronizing register 480to latch the signals at its input terminals. The assembly synchronizingregister 480 transmits its contents to the bus interface as RCV WORD(34:0) received word signals, comprising the thirty-two bits fromnibbles 480(7) through 480(0), a BOM LAT beginning-of-message latchedsignal, and an AFD LAT all-fall-down latched signal.

It will be appreciated that multiplexer 484 is provided to reduce theamount of time required to direct the respective signals to all of thenibbles 480(i) of the assembly synchronizing register 480 when it isforming the first word from a data router message packet 30. Inparticular, since the gate 482(6) is the last to be enabled for theword, the multiplexer 484 ensures that the signals from gate 481(6) donot have to be latched in buffer 482(6) before they are coupled to theinput terminals of nibble 480(6). This can reduce the amount of timerequired to assemble signals at the input terminals of all of thenibbles 481(6) by approximately one tick of the NODE CLK signal.

After the assembly synchronizing register 480 has transmitted RCV WORD(34:0) signals defining the first word of the data router messagepacket, the assembly control circuit generates the timing and controlsignal to enable the assembly synchronizing register 480 to assemble oneor more thirty-two bit data words, the number corresponding to thepreviously-stored length information. In assembling each word, theassembly control circuit 477 asserts a DATA LOOP signal and successivelyasserts the FLIT 7 OF WORD through FLIT 1 OF WORD signals. In response,the successive gates 481(7) through 481(0) couple the BUF SEL DATA (3:0)signals, which at successive ticks of the NODE CLK signal representcontents of successive data flits 36 for latching in the successivebuffers 482(7) through 482(1).

When the FLIT 0 OF WORD signal is asserted, the coincidence of theassertions of that signal and the asserted DATA LOOP signal enable anAND gate 492 to assert a DATA NIB (0) data nibble signal, which enablesmultiplexer 483 to couple the output signals from gate 481(0) to thenibble 480(0) of the assembly synchronizing register 480. In addition,contemporaneous with the assertion of the FLIT 0 OF WORD signal, theassembly control circuit 477 asserts the LAT WORD signal to enable gates485(7) through 485(1) to couple the contents of buffers 482(7) through482(1) to respective nibbles 480(7) through 480(1) of the assemblysynchronizing register 480. The assembly control circuit 477 alsoasserts the VALID WORD signal to notify the bus interface 453 that itmay assert the WRITE ASR write assembly synchronizing register signal toenable the assembly synchronizing register 480 to latch the signalsinput thereto.

It will be appreciated that multiplexer 483 is provided to reduce theamount of time required to direct the respective signals to all of thenibbles 480(i) of the assembly synchronizing register 480 when it isforming words from the data flits 36. In particular, since the gate481(0) is the last to be enabled for a word, the multiplexer 483 ensuresthat the signals from gate 481(0) do not have to be latched in buffer482(0) before they are coupled to the input terminals of nibble 480(0).This can reduce the amount of time required to assemble signals at theinput terminals of all of the nibbles 481(0) by approximately one tickof the NODE CLK signal.

The assembly control circuit 477 iteratively enables these operations tooccur until the assembly synchronizing register 480 has formed datawords from all of the data flits 32 in the data router message packet30. Thereafter, the assembly control circuit enables the assemblysynchronizing register 480 to form one last data word, which includesthe check signals generated by the message check generator 464 (FIG.9C-2), which are latched in the nibble 480(7) of the assemblysynchronizing register 480.

If the bus interface 452 is unable to accept a word from the assemblysynchronizing register 480 when the assembly control circuit 477 assertsthe VALID WORD signal, it may assert the STALL signal, which, asdescribed above, stalls the ejector first-in first-out buffer fromtransmitting signals representing successive flits to the messageassembler. Accordingly, the contents of the flit buffer 476 remainunchanged while the STALL signal is asserted. Thus, if the BUF SEL DATA(3:0) buffered selected data signals represent the tag field 35 if themessage assembler 452 is assembling the first word from the data routermessage packet, or data flits 36 to be loaded in the low-order nibble480(0) of the assembly synchronizing register 480 if it is assembling aword from the data flits 36, the flit buffer 476 maintains the signalswhile the STALL signal is asserted. Accordingly, the signals will nothave to be buffered in the respective buffers 482(6) or 482(2).

It will be appreciated that the first word formed in response to a newdata router message packet 30 includes the contents of the header field40 of a received data router message packet 30. It will be appreciatedthat, the processor 200, when it loads the send first register 234, 296or 303 to enable transmission of a new data router message packet, doesnot supply a value for the header field 40, since, as described above,the message injector port 223 generates the value itself. However, themessage assembler 452 keeps the value of the header field 40 of areceived data router message packet 30. If the message packet 30 wasreceived when the data router 15 is in all-fall-down mode, the value ofthe header field 40, which is loaded into the nibble 480(7) duringcreation of the first word, is used to identify the ones of nibbles480(4) through 480(0) which contain values from flits of the down pathidentification portion 41. If the value from the header field 40 werenot kept, the contents of nibbles 480(4) through 480(0) would be zeroedor purged at some point before the down path identification portion 41were loaded therein. The processor 200, when it retransmits the datarouter message packet, can use the header information stored in thenibble 480(7) to identify the number of valid flits for the down pathidentification portion 41.

FIGS. 9C-4 and 9C-5 depict logic diagrams of some elements of theassembly control circuit 477. FIG. 9C-4 depicts state machine circuitry,that generates the LEN length, TAG, DNF down flit and DATA LOOP dataloop signals. FIG. 9C-5 depicts circuitry that generates the FLIT "i" OFWORD signals. With reference first to FIG. 9C-4, initially a flip-flop500 is set, thereby asserting an IDLE ST idle state signal. If the SELDATA (4) (HEADER) signal is negated, an inverter 501 maintains an ANDgate 501 in an energized condition, which, in turn, enables an OR gate503 to assert an IDLE signal, which is latched by the flip-flop 500 ateach tick of the NODE CLK signal. Thus, while the SEL DATA (4) (HEADER)signal is negated, the IDLE signal remains asserted, enabling theflip-flop 500 to maintain the IDLE ST idle state signal asserted.

The asserted IDLE ST idle state signal also energizes an OR gate 504,which, in turn, enables one input terminal of an AND gate 505. If theSEL DATA (4) (HEADER) signal is asserted, the AND gate 505 is energizedto assert a HEADER signal. In response to the next tick of the NODE CLKsignal, a flip-flop 506 is set to assert a HEADER ST header statesignal. Contemporaneously, the asserted SEL DATA (4) (HEADER) signalenables inverter 501 to disable AND gate 502, in turn enabling the ORgate 503 to negate the IDLE signal. At the same tick of the NODE CLKsignal, the flip-flop 500 is reset, negating the IDLE ST idle statesignal.

The HEADER ST header state signal is asserted contemporaneously with thelatching by the flit buffer 476 (FIG. 9C-3) of SEL DATA selected datasignals corresponding to the contents of the header field 40 of the datarouter message packet 30 being received. Turning to FIG. 9C-5, theasserted HEADER ST signal energizes an OR gate 510 to assert an OTHER(7) signal, that, in turn, enables an OR gate 511 to assert the FLIT 7OF WORD signal. As described above, this enables gate 481(7) to couplethe BUF SEL DATA (3:0) buffered selected data signals, which at thatpoint correspond to the contents of the header field 40, to the buffer482(7).

Returning to FIG. 9C-4, it will be appreciated that the AND gate 505will assert the HEADER signal only while the SEL DATA (4) (HEADER)signal is asserted, which is the case only for one tick of the NODE CLKsignal. When the SEL DATA (4) (HEADER) signal is negated, the AND gate505 negates the HEADER signal, the flip-flop 506 is reset at the nexttick of the NODE CLK signal, negating the HEADER ST header state signal.As a result, the OTHER (7) signal generated by OR gate 510 (FIG. 9C-5)is negated, thereby enabling the OR gate 511 to negate the FLIT 7 OFWORD signal.

While the HEADER ST header state signal is asserted, it also enables oneinput terminal of an AND gate 512 and of a second AND gate 531. A thirdAND gate 514 receives the BUF SEL DATA (3:0) buffered selected datasignals, which still correspond to the contents of the header field 40.The AND gate 514 effectively determines whether the ejector port's leaf21 is the destination or if the message packet 30 is being receivedwhile the data router 15 is operating in an all-fall-down mode. Asdescribed above, if the data router 15 is not operating in anall-fall-down mode, the BUF SEL DAT (3:0) signals are all asserted, inwhich case the AND gate 514 is energized to assert a FLIT V=15 flitvalue equals fifteen signal. On the other hand, if the data router 15 isoperating in an all-fall-down mode, the AND gate is disabled, negatingthe FLIT V=15 signal.

If the FLIT V=15 flit value equals fifteen signal is negated, the secondinput terminal of AND gate 532 is disabled. However, the negated FLITV=15 signal enables an inverter 515 to, in turn, enable the second inputterminal of AND gate 512. The coincidence of the asserted HEADER STheader state and FLIT V=15 signals energizes the AND gate 512, enablingit to assert a LOAD DNF CTR load down flit counter signal. It will beappreciated that the AND gate 514 will assert the FLIT V=15 signalwhenever all of the BUF SEL DATA (3:0) signals are asserted, which mayalso occur when signals represent other fields of the data routermessage packet 30. However, the AND gate 512 will assert the LOAD DNFCTR load down flit counter signal only when the HEADER ST header statesignal is asserted, which will occur when the BUF SEL DATA (3:0) signalsrepresent the header field 40. The asserted LOAD DNF CTR load down flitcounter signal enables an OR gate 513 to assert a DNF LOOP down flitloop signal. In response to the next tick of the NODE CLK signal, aflip-flop 514 is set, enabling a DNF LOOP ST down flit loop state to beasserted. It will be appreciated that the HEADER ST header state signalwill at this point be negated.

At this point, a DNF CTR 0 down flit counter equals zero signal isasserted, which enables an inverter 525 to disable an AND gate 526.Returning to FIG. 9C-5, the BUF SEL DATA (3:0) buffered selected datasignals, which at this point represent the header 40 of the data routermessage packet 30 being received, are also coupled to a decrementationcircuit 515. The decrementation circuit 515 receives the four BUF SELDATA (3:0) signals and generates four signals representing abinary-encoded value that is one less than the binary-encoded value ofthe BUF SEL DATA (3:0) signals, and transmits the three high-ordersignals as LD VAL (3:1) load value signals to the initial data inputterminals of a counter 516. The asserted LD DNF CTR load down flitcounter signal enables the counter 516 to load the LD VAL (3:1) signalsas an initial count value.

The counter 516 transmits DNF CNT (3:0) down flit count signals,decrementing their binary-encoded value in response to successive ticksof the NODE CLK signal. The DNF CNT (3:0) signals are coupled to aninverter 523 that couples the complements of each of the signals toinput terminals of an AND gate 524. If all of the DNF CNT (2:0) signalsare negated, which occurs if the binary-encoded encoded value equalszero, the AND gate 524 asserts a DNF CTR 0 down flit counter equals zerosignal. However, when the counter 516 generates DNF CNT (2:0) down flitcount signals having other binary-encoded values, the inverter 523 willdisable the AND gate 524 to negate the DNF CTR 0 down flit countersignal.

The negated DNF CTR 0 down flit counter signal is coupled to an inverter525 (FIG. 9C-4) which enables one input terminal of an AND gate 526.Since the other input terminal of AND gate 526 is then enabled by theasserted DNF LOOP ST down flit loop state signal, the AND gate 526 isenergized. The energized AND gate 526 enables the OR gate to maintainthe DNF LOOP down flit loop signal in an asserted condition, which, inturn, causes the flip-flop 514 to remain set during succeeding ticks ofthe NODE CLK signal. This, in turn, maintains the DNF LOOP ST down flitloop state signal in an asserted condition. Thus, the DNF LOOP ST downflit loop state signal remains asserted until the DNF CTR 0 down flitcounter equals zero signal is asserted, as described below, which causesthe inverter 525 to disable AND gate 526.

In addition, the DNF CNT (2:0) down flit count signals are coupled to adecoder 517 that asserts a respective one of eight DEC DNF CNT (7:0)decoded down flit count signals associated with the binary-encoded valueof the DNF CNT (2:0) down flit count signals. The DEC DNF CNT (4:0)decoded down flit count signals are coupled to respective ones of flit"i" of word signal generating circuits, one of which, identified byreference numeral 520(i), is shown in FIG. 9C-5. Circuit 520(i) includesan AND gate 521 that is enabled by the asserted DNF LOOP ST down flitloop state signal from flip-flop 514 (FIG. 9C-4). When the decoder 517asserts the particular DEC DNF CNT (i) decoded down flit signalassociated with the circuit 520(i), the AND gate 521(i) is energized toassert a DNF (i) OF WORD down flit "i" of word signal. This signal, whenasserted, in turn energizes an OR gate 522(i), enabling it to assert theFLIT "i" OF WORD signal.

In one embodiment, in which the down path identification portion 41 of adata router message packet 30 may have five flits, only the fivelow-order DEC DNF CNT (4:0) decoded down flit count signals may beasserted. In addition, the assembly control circuit 477 includes fivecircuits 520(i), each associated with one of the DEC DNF CNT (4:0)signals. The DNF LOOP ST down flit loop state signal enables AND gates521(i) in all of the circuits 520(i) in parallel. However, the decoder517 asserts only one DEC DNF CNT "i" decoded down flit signal at a time,which in turn determines the FLIT "i" OF WORD signal asserted. The FLIT"i" OF WORD signals determine the ones of buffers 482(4) through 482(0)which will receive and latch the BUF SEL DATA (3:0) signals, at each ofthe successive ticks of the NODE CLK signal, as described above.

At some point in the decrementing of counter 517, the binary-encodedvalue of the DNF CNT (2:0) down flit count signals from counter 516 willequal zero. When that occurs, the signals are all negated, in which casethe inverter 523 enables all input terminals of an AND gate 524, whichis energized to assert the DNF CTR 0 down flit counter equals zerosignal. The asserted DNF CTR 0 signal causes the inverter 525 (FIG.9C-4) to disable AND gate 525, and causing the OR gate 513 to negate theDNF LOOP down flit loop signal. The negated DNF LOOP signal causes theflip-flop 514 to be reset at the next tick of the NODE CLK signal. Atthis point, if the data router message packet 30 being received has anyflits in the down path identification portion 41, they have been latchedin the respective buffers 482(4) through 482(0).

If the data router message packet 30 being received has flits in thedown path identification portion 41, the asserted DNF LOOP ST down flitloop state signal also enables one input terminal of an AND gate 527.When the DNF CTR 0 down flit counter equals zero signal is asserted asdescribed above, the AND gate 527 is energized, which, in turn,energizes an OR gate 530 to assert a LEN length signal. The asserted LENlength signal enables a flip-flop 531 to be set in response to the nexttick of the NODE CLK signal, enabling the assertion of a LEN ST lengthstate signal. It will be appreciated that when the LEN ST length statesignal is asserted, the BUF SEL DATA (3:0) buffered selected datasignals correspond to the contents of the message length field 34 of thedata router message packet being received.

Alternatively, if the down path identification portion 41 of the datarouter message packet 30 is empty, when the HEADER ST header statesignal is asserted the BUF SEL DATA (3:0) buffer selected data signalswill all be asserted. The asserted BUF SEL DATA (3:0) signals enable ANDgate 514 to assert the FLIT V=15 flit value equals fifteen signal. Theasserted FLIT V=15 signal causes inverter 515 to disable AND gate 512,which, in turn, prevents flip-flop 514 from being set. The disabled ANDgate 512 maintains the LOAD DNF CTR load down flit counter signal in anegated condition, keeping the counter 516 (FIG. 9C-5) from operating.

The asserted FLIT V=15 flit value equals fifteen signal also enables oneinput terminal of an AND gate 532. Since the flip-flop 506 at that pointis asserting the HEADER ST header state signal, AND gate 532 isenergized, which energizes the second input terminal of OR gate 530 toassert the LEN signal. The asserted LEN signal enables the flip-flop 531to be set in response to the next tick of the NODE CLK signal, enablingthe assertion of a LEN ST length state signal. It will be appreciatedthat the LEN ST length state signal is asserted one tick of the NODE CLKsignal after assertion of the HEADER ST header state signal by flip-flop506, indicating that the BUF SEL DATA (3:0) buffered selected datasignals represent the message length field 35 one tick of the NODE CLKsignal after they represent the header field 40, as would be the case ifthe down path identification portion 41 of the data router messagepacket 30 being received is empty. Immediately after the flip-flop 531has been set, the HEADER ST header state signal is negated, therebyenabling AND gate 532 and OR gate 320 to negate the LEN length signal.

At the next tick of the NODE CLK signal, the asserted LEN ST signalenables a flip-flop 529 to be set to assert a TAG ST tag state signal.Contemporaneously, since the LEN length signal is negated, the flip-flop531 is reset to negate the LEN ST length state signal. The asserted TAGST tag state signal energizes an OR gate 532 to assert a DATA LOOPsignal, which enables a flip-flop 533 to be set in response to the nextNODE CLK node clock signal to assert a DATA LOOP ST data loop statesignal. In response to the same tick of the NODE CLK signal, the negatedLEN ST length state signal enables the flip-flop 529 to be reset,negating the TAG ST tag state signal.

Accordingly, it will be appreciated that flip-flops 531, 529 and 533 areset in response to sequential ticks of the NODE CLK signal, tosequentially assert the LEN ST length state, TAG ST tag state and DATALOOP ST data loop state signal. In addition, the flip-flops 531 and 529are reset after being set for one tick, so that the LEN ST and TAG STsignals are only asserted for one tick of the NODE CLK signal. It willbe appreciated that while the LEN ST and TAG ST signals are asserted,the BUF SEL DATA (3:0) signals represent, consecutively, the contents ofthe length and tag fields 34 and 35 of the data router message packet 30being received.

In addition, a DATA CTR EQ 0 data counter equals zero signal isinitially negated, which, in turn, enables an inverter 538 to enable oneinput terminal of an AND gate 539. The asserted DATA LOOP ST data loopstate signal energizes the AND gate 539, which enables the OR gate 532to maintain the DATA LOOP signal in an asserted condition. Accordingly,the flip-flop 533 will remain in a set condition, maintaining the DATALOOP ST data loop state signal asserted. As will be described in moredetail below, the DATA CTR EQ 0 signal is asserted when the BUF SEL DATA(3:0) signals represent the last data flit 36 of the data router messagepacket 30 being received. At that point, the asserted DATA CTR EQ 0signal disables AND gate 539, enabling OR gate 532 to negate the DATALOOP signal and causing the flip-flop 533 to be reset, negating the DATALOOP ST signal, in response to the next tick of the NODE CLK signal.

Returning to FIG. 9C-5, the LEN ST length state and TAG ST tag statesignals are coupled to respective ones of OR gates 534 and 535, toenable them to assert the FLIT 5 OF WORD and FLIT 6 OF WORD signals,respectively. This enables the BUF SEL DATA (3:0) buffered selected datasignals, which sequentially correspond to the contents of the messagelength and message tag fields 34 and 35 to be directed to and latched byrespective buffers 482(5) and 482(6).

The LEN ST length state signal is also directed to a data flit countercircuit (FIG. 9C-5) as a LOAD DATA CTR load data counter signal. Whenthe LOAD DATA CTR load data counter signal is asserted, it enables afour-bit binary word counter 536 to load the BUF SEL DATA (3:0) bufferedselected data signals, which at this point represent the contents of themessage length field 34 of the data router message packet 30 beingreceived. The asserted LOAD DATA CTR load data counter signal alsoenergizes an OR gate 537, which, in turn, enables a three-bitflits-per-word binary counter 540 to load an initialization value.

As described above, the value in the message length field 34 representsthe number of thirty-two bit words in the four-bit data flits 36 in themessage packet 30. The initialization value loaded by flits-per-wordcounter 540 identifies the number of data flits 36 in each wordenumerated by the message length field 34. Since the BUF SEL DATA (3:0)buffered selected data signals at successive ticks of the NODE CLK nodeclock signal, after representing the message tag field 35, represent thedata flits 36, counter 34 is initially disabled by the TAG ST tag statesignal, and then decremented in response to successive ticks of the NODECLK signal.

The flits-per-word counter 540 generates binary-encoded FLIT/WRD CNT(2:0) flits-per-word count signals which are directed to a decoder 541.The decoder 541 generates DEC DATA FLIT CNT (7:0) decoded data flitcount signals, each of which is asserted in response to the associatedbinary value of the FLIT/WRD CNT (2:0) signals. The DEC FLIT/WRD CNT(7:0) signals are used in the generation of the FLIT "i" OF WORD signalsto enable the gates 481(i) and buffers 482(i) to assemble the particularportions of a thirty-two bit data word. In particular, each DEC FLIT/WRDCNT (i) signal is coupled to one input terminal of an AND gate 548(i)("i" is an integer between 7 and 0), all of which are enabled inparallel by the asserted DATA LOOP ST data loop state signal from theflip-flop 533 (FIG. 9C-4). While the DATA LOOP ST signal is asserted, asthe DEC FLIT/WRD CNT (7) through DEC FLIT/WRD CNT (0) signals aresequentially asserted, the AND gates 548(7) through 548(0) aresequentially energized.

When AND gate 548(7) is energized, it asserts a DATA FLIT 7 OF WORDsignal to energize the OR gate 511, which, in turn, asserts the FLIT 7OF WORD signal. When AND gate 487(6) is energized, it asserts a DATAFLIT 6 OF WORD signal to energize OR gate 535, which, in turn, assertsthe FLIT 6 OF WORD signal. Similarly, when AND gate 548(5) is energized,it asserts a DATA FLIT 5 OF WORD signal to energized OR gate 534, which,in turn, asserts the FLIT 5 OF WORD signal. In addition, each of thecircuits 520(i) includes an AND gate 548(i), which it energized when thecorresponding DEC DATA FLIT CNT (i) signal is asserted, enabling it toassert a DATA FLIT "i" OF WORD signal. The asserted signal energizes theOR gate 522(i), which, in turn, asserts the FLIT "i" OF WORD signal.Each of the FLIT "i" OF WORD signals enables the gates 481(i) andbuffers 482(i) to successively gate and latch BUF SEL DATA (3:0)buffered selected data signals representing successive data flits 36 toform a thirty-two bit word, as described above.

The FLIT/WRD CNT (2:0) flits-per-word count signals are also directed toan inverter 542, which couples complemented signals to an AND gate 543.When the binary-encoded value of the FLIT/WRD CNT (2:0) signals is zero,all of the complemented signals are asserted and inverter 542 energizesthe AND gate 543 to assert a FLIT/WRD CNT 0 flit-per-word count equalszero signal. When the FLIT/WRD CNT 0 signal is asserted, the BUF SELDATA (3:0) buffered selected data signals corresponds to the low-ordernibble of a thirty-two bit data word. The asserted FLIT/WRD CNT 0 signalenables one input terminal of an AND gate 544. If a WORD CNT 0 wordcount equals zero signal, which is generated in response to the currentvalue of the word counter 536, is not asserted, AND gate 544 isde-energized to negate the DATA CTR EQ 0 data counter equal zero signal.As will be described in more detail below, when the WORD CNT 0 wordcount equals zero signal is asserted, the thirty-two bit word currentlybeing formed by the gates 481(i) and buffers 482(i) is the last in thedata router message packet 30 being received.

The FLIT/WRD CNT 0 flits-per-word count equals zero signal is coupled toan input terminal of an AND gate 545. In addition, in response to thenext tick of the NODE CLK signal after assertion of the FLIT/WRD CNT 0signal, the AND gate 545 is energized to assert a WORD DOWN signal,which is used to form the asserted LAT WORD latch word and VALID WORDsignals. The asserted WORD DOWN signal energizes the OR gate 537, which,in turn, enables the flits-per-word counter 540 to reload. In addition,the asserted WORD DOWN signal enables the word counter 536 to decrement.

The word counter 536 generates binary-encoded WORD CNT (4:0) word countsignals that identify the number of thirty-two bit data words that havebeen received. An inverter 546 couples complemented WORD CNT (4:0)signals to the input terminals of an AND gate 547. If all of the WORDCNT (4:0) signals are negated, which will occur when the BUF SEL DATA(3:0) buffered selected data signal represent the data flits 36 of thelast thirty-two bit word in the data router message packet 30. When theFLIT/WRD CNT 0 flits-per-word count equals zero signal is also asserted,which will occur when BUF SEL DATA (3:0) signals represent the last dataflit 36 of the last thirty-two bit word, the AND gate 544 is energizedto assert the DATA CTR EQ 0 data counter equals zero signal.

Returning to FIG. 9C-4, the assertion of the DATA CTR EQ 0 data counterequals zero signal enables inverter 538 to disable AND gate 539, whichde-energizes OR gate 532 causing the DATA LOOP signal to be negated. Onthe other hand, since the DATA LOOP ST data loop state signal isasserted, the assertion of the DATA CTR EQ 0 signal energizes an ANDgate 550 to assert a CHECK signal. In response to the next tick of theNODE CLK signal, the negated DATA LOOP signal will enable the flip-flop533 to be reset, negating the DATA LOOP ST signal, and the flip-flop 551to be set, asserting a CHECK ST signal. Since at this point, the DATALOOP ST signal is negated, the AND gate 550 is disabled thereby negatingthe CHECK signal. The negated CHECK signal causes the flip-flop 551 tobe reset in response to the next tick of the NODE CLK signal, therebynegating the CHECK ST signal. Accordingly, the CHECK ST signal isasserted for only one tick of the NODE CLK signal.

It will be appreciated that the CHECK ST signal is asserted when the BUFSEL DATA (3:0) signals represent the check Bits from the check generator464 (FIG. 9C-2). The CHECK ST signal is coupled to OR gate 510 (FIG.9C-5) which is energized to assert the OTHER 7 signal. This signal, asdescribed above, enables the OR gate 511 to assert the FLIT 7 OF WORDsignal, enabling the gate 481(7) to direct the signals to the buffer482(7), which latches them as described above.

The CHECK ST signal is also coupled to an AND gate 552. If the SEL DATA(4) (HEADER) signal is negated, an inverter 553 enables one inputterminal of the AND gate 552. The assertion of the CHECK ST signalenergizes the AND gate, which, in turn, energizes the OR gate 503,enabling it to assert the IDLE signal. In response to the next tick ofthe NODE CLK signal, the flip-flop 500 is set to assert the IDLE ST idlestate signal. At this point, the sequence of operations described abovein connection with FIGS. 9C-4 and 9C-5 can be repeated.

If, on the other hand, the SEL DATA (4) (HEADER) signal is asserted,which can occur if the FIFO 451 is transmitting FIFO OUT (4:0) signalsrepresenting the first flit of a new data router message packet 30, atthe same time the BUF SEL DATA (3:0) signals represent the last flit ofa current data router message packet 30, the inverter 553 maintains theAND gate 552 in a de-energized condition. If that occurs, the assertedCHECK ST signal energizes OR gate 504, which enables one input terminalof AND gate 505. The asserted SEL DATA (4) (HEADER) signal energizes theAND gate 505, enabling it to assert the HEADER signal, which setsflip-flop 506 at the next tick of the NODE CLK signal, enabling theoperations described above to be performed without requiring assertionof the IDLE ST signal. This allows the message assembler 452 to beginprocessing the new message packet 30 one tick of the NODE CLK signalearlier than would be the case if the CHECK ST signal only operated toenable setting of flip-flop 500.

Returning to FIG. 9C-3, as described above, after the bus interface 453has been notified by the assertion of the VALID WORD signal that a newword is available for latching in the assembly synchronizing register480, it may assert the WRITE ASR write assembly synchronizing registersignal. The asserted WRITE ASR signal enables the register 480 to latchthe signals in the respective nibbles 480(i), in the beginning ofmessage flag 486 and in the all-fall-down flag 48. The assemblysynchronizing register 480 transmits the latched signals to the businterface 453 as RCV WORD (34:0) signals, comprising a includethirty-two bit data word as the RCV WORD (32:0) signals, a BOM LATlatched beginning-of-message signal as the RCV WORD (33) signal, and anAFD LAT latched all-fall-down mode signal as the RCV WORD (34) signal.

FIG. 9C-6 depicts a detailed logic diagram of the bus interface 453 inone embodiment of the data router interface 205. With reference to FIG.9C-6, the bus interface includes two first-in first-out buffers (FIFOs)560 and 561, both of which are connected to receive selected portions ofthe RCV WORD (34:0) received word signals from the assemblysynchronizing register 480, and a length store 562, which receivesselected portions of the REV DATA (34:0) signals from a multiplexer 563.

FIFO 560 receives the length and tag portions, comprising the RCV WORD(28:22) signals from nibbles 480(5) and 480(6) respectively, and the AFDLAT all-fall-down latch signal from all-fall-down flag 487 of theassembly synchronizing register 480. The FIFO 560 is used to control thecontents of the received length field 253, received tag field 245, andreceived all-fall-down mode field 254 of the statues and privateregisters 293 and 294 of left interface register set 290, or of statusand private registers 300 and 301 of right interface register set 291(FIG. 9A-2B).

If (i) the FIFO 560 is not asserting a ST INF FIFO NR FULL statusinformation buffer nearly full signal, which indicates whether the FIFO560 is able to buffer additional information, (ii) the FIFO 561 is notasserting a DA INF FIFO NR FULL data information buffer nearly fullsignal, and (Hi) a REC EN receive enable signal is asserted, a controlcircuit 564 generates a WRT write signal to energize one input terminalof an AND gate 565. When the RCV WORD receive word signals represent thefirst word of a data router message packet 30, they have the length, tagand all-fall-down mode information. In that case, the BOM LAT latchedbeginning-of-message signal, which comprises the RCV WORD (33) signal,is asserted, which energizes the second input terminal of an AND gate564, enabling it to assert a STA WRT EN status write enable signal. Whenthe STA WRT EN signal is asserted, the FIFO 560 latches the portion ofthe RCV DATA (34:0) signals representing the length, tag andall-fall-down mode information at the next tick of the NODE CLK signal.

The data first-in first-out FIFO 561 is used to buffer the RCV WORD(31:0) signals from the assembly synchronizing register 480. The FIFO561 is used to control the contents of the receive registers 233, 295and 302 of the middle, left and right register sets 230, 290 and 291(FIGS. 9-2A and 9A-2B).

The BOM LAT latched beginning-of-message signal and AFD LAT latchedall-fall-down mode signal are used, along with the WRT write enablesignal from control circuit 564, to control buffering of the RCV WORD(32:0) received word signals, by FIFO 561. If a normally-negated RCVSTOP receive stop signal is negated, a multiplexer 566 couples the RCVWORD (32:0) receive word signals to the data input terminal of the FIFO561. If the AFD LAT latched all-fall-down mode is negated indicatingthat the data router message packet 30 was not received while datarouter 15 was in all-fall-down mode, the BOM LAT and WRT signals controlstorage of the RCV WORD (32:0) signals in the FIFO 561. In that case,when the BOM LAT signal is asserted, which occurs when the RCV WORD(32:0) receive word signals define the length, tag and all-fall-downmode information, an inverter 578 maintains an OR gate 567 in ade-energized condition, causing it to disable one input terminal of anAND gate 570.

The AND gate 570 remains disabled regardless of the assertion level ofthe WRT write enable signal, in turn disabling one input terminal of anOR gate 571. If a PEI BUS WRT DA FIFO interface bus write data FIFObuffer signal is negated, which is the case unless the processor 200 isattempting to load the data words into the first-in first-out FIFO 561,the OR gate 571 remains disabled and negates a DA WRT EN data FIFObuffer write enable signal, inhibiting the FIFO 561 from loading the RCVWORD (32:0) signals.

For each succeeding word derived from the message packet 30, the BOM LATlatched beginning-of-message signal will be negated. In that case, theinverter 568 will energize the OR gate 567, enabling it to enable oneinput terminal of AND gate 570. In response to the assertion of the WRTwrite enable signal from control circuit 564, the AND gate will beasserted, in turn energizing OR gate 571. The energized OR gate 571asserts the DA WRT EN data write enable signals. While the DA WRT ENsignal is asserted, the data first-in first-out FIFO 561 latches thenext word of the message packet 30.

If the FIFO 561 becomes nearly full, it can assert the DA FIFO NR FULLnearly full signal, which disables the control circuit 564 fromasserting the WRITE ASR write assembly synchronizing register signal,and inhibiting it from asserting the WRT signal in response to receiptof a VALID WORD signal from the message assembler 452. In addition, thecontrol circuit 564 will assert the STALL signal to stall the messageassembler 452 as described above.

It will be appreciated that the concurrent assertion of the BOM LATlatched beginning-of-message signal and negation of the AFD LAT latchedall-fall-down mode signal inhibits the data first-in first-out FIFO 561from latching the first word of a new message packet 30. This conditionwill occur when the message packet 30 is received while the data router15 is not in all-fall-down mode, in which case the address information,defined by the RCV WORD (31:28) and RCV WORD (19:0) receive word signalsneed not be retained. Further, the remaining information, namely, thelength and tag information represented by the RCV WORD (27:20) receiveword signals, as well as the AFD LAT latched all-fall-down mode signal,are available in the FIFO 560 and the appropriate status register. Byinhibiting the data first-in first-out FIFO 561 from latching the firstword assembled by the message assembler 452 from a data router messagepacket 30 that was received when the data router 15 is not inall-fall-down mode, the number of words that the processor 200 mustretrieve from the FIFO 561 for a particular message packet 30 isreduced. Otherwise stated, this enables the FIFO 561 to hold more wordsformed from data flits 36 of the received message packets 30.

On the other hand, if the data router message packet 30 is beingreceived while the data router 15 is in all-fall-down mode, the firstword of the RCV WORD (31:0) receive word signals is buffered in the FIFO561. Enabling the word to be buffered in the FIFO 561 permits theaddress information, defined by the RCV WORD (31:28) and RCV WORD (19:0)signals to be retained, which will facilitate retransmission of themessage packet 30 through the data router interface 205 as describedabove. In that case, the asserted AFD LAT latched all-fall-down modesignal energizes OR gate 567, which enables one input terminal of ANDgate 570. The AND gate 570 is energized in response to the asserted WRTsignal as described above, Which energizes OR gate 571 to assert the DAWRT EN data buffer write enable signal, also as described above. It willbe appreciated that the data FIFO 561 will latch the RCV WORD (31:0)signals defining the first word of the data router message packet 30 atthe same time the status FIFO 560 is latching the RCV WORD (27:20) andRCV WORD (33) signals defining the length, tag and all-fall-down modeinformation for the same word. The data first-in first-out FIFO 561buffers RCV WORD (32:0) signals defining succeeding words of the datarouter message packet 30 in the same manner as described above

The contents of the length store 562 identify the number of thirty-twobit data words of the data router message packet 30 remaining to bereceived and buffered in the FIFO 561. The asserted BOM LAT latchedbeginning-of-message signal also enables the multiplexer 563 to couplethe portion of the RCV DATA (34:0) signals representing the lengthinformation to the input terminals of the length store 562. The assertedWRT write enable signal enables the length store 562 to latch thesignals at its input terminals, which at this point identify the lengthof the data router message packet 30 being received.

The LEN CNT length count output signals from length store 562 arecoupled to the input terminals of a decrementation circuit 572, whichgenerates NXT LEN CNT next length count signals that have abinary-encoded value one less than the binary-encoded value of the LENCNT length count signals. With successive words of the RCV WORD receiveword signals, the BOM LAT latched beginning-of-message signal isasserted, enabling multiplexer 563 to couple the NXT LEN CNT next lengthcount signals to the input terminal of length store 562, which latchesthem in response to the asserted WRT write enable signal. Since thesuccessive assertions of the WRT write enable signal also control theloading of RCV WORD (31:0) receive word signals defining successivewords in the data router message packet being received, the length storeis successively decremented for each word latched in the FIFO 561.

Further, since the value initially loaded into the length store 562identifies the number of words in the data router message packet 30being received, when the contents of the length store 562 go to zero,all of the data words will be in the FIFO 561. When that occurs, the LENCNT length count signals are all negated, defining a binary-encodedvalue of zero. The LEN CNT length count signals from length store 562are coupled to a NOR gate 573, which generates an asserted LEN 0 lengthequals zero signal when all of the LEN CNT signals are negated.

If the first-in first-out FIFO 560 is asserting the ST INF FIFO NEstatus information first-in first-out buffer not empty signal, it andthe asserted LEN 0 length count equals zero signal energize an AND gate574, which, in turn, energizes an OR gate 575 to assert an NEW DR MSGnew data router message signal. The asserted NE DR MSG signal indicatesthat the data router interface 205 has received a new data routermessage packet 30 from the data router 15. The signal may be used tocondition the receive bit 241 of the corresponding status register 231,293 or 300 (FIGS. 9A-2A and 9A-2B), and may also enable the networkinterface 202 to interrupt the processor 200, in turn enabling theprocessor 200 to retrieve the data router message packet 30.

In addition, the asserted NEW DR MSG new data router message signalenables one input terminal of an AND gate 576. When the other inputterminal of AND gate 576 is also enabled by an EN RD ST FIFO enable readstatus first-in first-out buffer signal, the AND gate 576 asserts a RDST FIFO read status first-in first-out buffer signal. The state of theEN RD ST FIFO signal is effectively controlled by circuitry controllingthe status register 231, 293 or 300, and is asserted by theinjector/ejector common control/status circuit 222 when the processor200 has finished retrieving a previously-received message packet 30whose length, tag and all-fall-down mode were in the status register. Atthat point, the circuitry controlling the status register can enable theinformation from the FIFO 560 to be loaded into the appropriate fieldsof the status register, and so it asserts the EN RD ST FIFO signal,enabling the AND gate to assert the RD ST FIFO signal.

Retrieving the contents of the data first-in first-out FIFO 561 iscontrolled by a READ DA FIFO read data first-in first-out buffer signal.This signal is asserted by the ejector common control/status circuit 226when the processor 200 is reading the respective receive register 233.

The bus interface 453 also includes several other facilities. First, itwill be appreciated that, at the point of a context switch operation thedata first-in first-out FIFO 561 may contain data router message packetdata. In the performing the context switch, it is necessary to drain thecontents of the data first-in first-out FIFO 561 so that data of messagepackets that were transferred in one context are retrieved by theprocessor 200 and processed in that context, and not in some othercontext. The processor 200 can perform that operation by retrieving thedata through the receive registers 260, 295 and 302. On the other hand,when the context is restored, it will be necessary to load the datawords that were in the FIFO 561 at the point of the context switch backinto the FIFO 561. This will restore the context to the same conditionas at the point of the context switch.

To accommodate that, the second input terminal of multiplexer 566 isconnected to receive data from the interface bus 211. In this condition,the RCV STOP receive stop signal is asserted, enabling the multiplexerto couple the data from the bus 211 to the input terminal of thefirst-in first-out FIFO 561. In addition, the interface 212 asserts aPEI BUS WRT DA FIFO write data first-in first-out buffer signal thatenergizes OR gate 571 to assert the DA WRT EN data first-in first-outbuffer write enable signal. This enables the first-in first-out FIFO 561to load the data from the interface bus 211, enabling the processor 200to restore the context to the point at which the context switchoccurred.

It will be appreciated that the processor 200 may need to load data intothe first-in first-out buffer 562 for other reasons, as well. Forexample, the processor 200 may need to perform a loop-back test, inwhich it loads data into the FIFO 561, and reads it back to verifyproper working of the data router interface 205. This facility permitsit to do that.

Another facility allows the processor 200 to control disabling ofreception of data router message packets 30 by the ejector port 225,while at the same time ensuring that the bus ejector port 225 receives acomplete data router message packet even if the processor 200 disablesreception while the port is receiving a message packet 30. As notedabove, the processor 200 can disable the ejector port 225 from receivingdata router message packets 30 by setting the receive stop bit 252 ofthe private register 233, or of corresponding bits of private registers295 or 301 of the left and right register sets 290 and 291,respectively.

In particular, as noted above when the bus interface 453 has receivedall of the data words off a data router message packet 30, the contentsof the length store 562 have the value zero. When that occurs, the LENCNT length count signals are all negated to represent the binary-encodedvalue zero, causing the NOR gate 573 to assert the LEN 0 length equalszero signal. In addition to controlling retrievals from FIFO 560 throughAND gate 574, the LEN 0 signal enables one input terminal of an AND gate577.

The second input terminal of AND gate 577 is controlled by a REC STOPREQ receive stop request signal from a receive stop control circuit 580,which is depicted in FIG. 9C-7. With reference to FIG. 9C-7, the receivestop control circuit 580 includes a flip-flop 581 which is set inresponse to an asserted SET RCV STOP REQ FF set receive stop requestflip-flop signal from an AND gate 582. The AND gate 582 is energized toassert the SET RCV STOP REQ FF signal in response to signals from theinterface 212 (FIG. 8) that control setting of the receive stop bit ofthe respective private registers 233, 290 or 291.

The asserted REC STOP REQ receive stop request signal energizes thesecond input terminal of AND gate 577. When the LEN 0 length equals zerosignal is also asserted, the AND gate is energized to assert the RECSTOP GRANT receive stop grant signal. An inverter 586 disables an ANDgate 587, causing to negate the REC EN receive enable signal to disablethe control circuit 564 from asserting the WRT write enable signal,effectively disabling the bus interface 453 from receiving from theassembly synchronizing register 480. Since the AND gate 577 is notenergized until the LEN 0 signal is asserted, the AND gate 587 is notdisabled until all of the data words of a data router message packet 30that was being received when the SET RCV STOP REQ FF set receive stoprequest flip-flop signal was asserted, have been loaded into the FIFO561.

The REC STOP GRANT receive stop grant signal from AND gate 577 is alsocoupled to an input terminal of an AND gate 585 of the receive stopcontrol circuit 580 (FIG. 9C-7). When the REC STOP GRANT signals havebeen asserted by both the left and right ejector ports 225(l) and225(r), an AND gate 585 is energized to assert a SET REC STOP setreceive stop signal, which resets flip-flop 581, which negates the RECSTOP REQ receive stop request signal, and sets a flip-flop 583 to asserta REC STOP receive stop signal.

Returning to FIG. 9C-6, the negated REC STOP REQ receive stop requestsignal disables AND gate 577, causing it to negate the REC STOP GRANTreceive stop grant signal. The inverter 586 complements this signal,thereby enabling one input terminal of AND gate 587. However, theasserted REC STOP receive stop signal from flip-flop 583 (FIG. 9C-7),through an inverter 590, causes the AND gate 587 to maintain the REC ENreceive enable signal in a negated condition.

The negated REC STOP GRANT signal also disables AND gate 585, causing itto negate the SET REC STOP set receive stop signal. The processor 200can thereafter enable the respective ejector port to resume receiving byclearing the receive stop bit of the respective private register. Indoing so, and AND gate 584 is energized to assert a RESET REC STOP resetreceive stop signal, enabling the flip-flop 583 to be reset, therebynegating the REC STOP receive stop signal. Returning to FIG. 9C-6, thenegated REC STOP receive stop signal is complemented by inverter 590 toenable the second input terminal of AND gate 587, thereby energizing itto assert the REC EN receive enable signal. Thereafter, the controlcircuit 564 can resume asserting WRITE ASR write assembly synchronizingregister signal and the WRT EN write enable signal to enable receipt ofdata router message packets 30 to be resumed.

As with the description of the message injector port 223 as noted above,in the preceding description of the message ejector port 225, all of thecircuits operate in response to the NODE CLK signal provided by theclock buffer 207, whereas it may be desirable to have most of theprocessing element 11, including most of the network interface 202, tooperate in response to a processing element clock signal (not shown)which synchronizes most operations on the processing element 11, andonly small portions of the network interface 202 operate in response tothe NODE CLK signal. In that situation, in one embodiment the flitreceiver stage 450, ejector FIFO 451 and ejector message assembler 452operate in response to the NODE CLK signal, whereas the bus interfacecircuit 453 operates in response to the PE CLK signal. In thatembodiment, the VALID WORD signal provided by the bus interface 453 isnot coupled directly to the ejector message assembler 452, but insteadis synchronized through a synchronizer circuit similar to thesynchronizer circuit 444 depicted in FIG. 9B-9.

iv. Status/Control Circuitry

FIGS. 9D-1 through 9D-7 depict details of circuitry controlling severalfields of the status registers 231, 293 and 300 (FIGS. 9A-2A and 9A-2B),and private registers 232, 294 and 301, as well as the message countregister 313. FIGS. 9D-1 through 9D-5 depicts details of circuitrycontrolling loading of the status and private registers in response toreceipt of data router message packets 30 from the data router 15 and inresponse to loading by the processor 200 (FIG. 8) over the interface bus211. It will be appreciated that loading of the respective registersunder control of the processor 200 enables the processor 200 toinitialize them, and also to establish context in response to a contextswitch. FIG. 9D-1 depicts a detailed block diagram, and FIGS. 9D-2through 9D-5 depicts detailed circuitry, for controlling loading of, andreading from, the "middle" registers, in a manner transparent to theprocessor 200,

With reference to FIG. 9D-1, the [LEFT] MSG STAT message status signalsfor a message received by the left message ejector port 225(l) arecoupled from the status information FIFO 560 over a bus 600. In responseto [LEFT] NEW STATUS signals from the bus interface 453, indicatingreceipt of a new data router message packet 30 to be retrieved by theprocessor 200, the [LEFT] MSG STAT signals are latched in various fieldsof the status register 293 and private register 294. In particular, afield comprising MSG [LEFT] STAT [REC LENGTH] signals, indicating thelength of the received message 30, are latched by a latch 601, whichprovides the receive length field of the status register 293. Inaddition, the [LEFT] MSG STAT [REC LENGTH] signals are coupled through aselection and decrementation circuit 602 and latched in a latch 603,which provides the receive length remaining field of the status register293. Similarly, the [LEFT] MSG STAT [TAG] signals, representative of thetag field of the received message packet 30, are latched in a latch 604,which provides the receive tag field of the status register 294.Finally, the [LEFT] MSG STAT [AFD] signals, which indicate whether themessage packet 30 is received while the data router 15 is inall-fall-down mode and that the message packet 30 has not arrived at itsdestination leaf 21, are latched in a latch 605, which provides thereceived all-fall-down flag of the private register 294. A similar busand set of latches (not shown) are provided for the right port ejector225(r).

The latches 601, 603, 604 and 605 can also be loaded from the interfacebus 211. In particular, the appropriate ones of the data lines carryingsignals, identified on FIG. 9D-1 as the PEI BUS [LEN], PEI BUS [TAG],and PEI BUS [AFD], respectively, are connected to the latches 601, 603,and 605, respectively. In addition, the data lines carrying signalsidentified as PEI BUS [LEN REM] are connected to the selection anddecrementation circuit. 602. In any case, the respective signals can belatched by the various latches in response to write enable signals,generally identified as PEI BUS WRT LDR REG write left data routerregisters signals from the interface 212 under control of the processor200. The interface 212, under control of the processor 200, can alsoenable the contents of the various latches to be coupled onto the bus211 in response to corresponding PEI BUS RD LDR REG register readenabling signals.

The selection and decrementation circuit 602 performs several functions.When either the [LEFT] MSG STAT signals or signals from bus 211 arebeing loaded into latch 603, it selectively couples the signals from oneor the other to the latch 603. In addition, when the processor 200 isreading from the receive register 295, it decrements the binary-encodedvalue represented by the signals latched in the latch 603 and enablesthe result to be latched therein, to represent the length of the messagepacket 30 remaining to be read. In addition, if the [LEFT] MSG STAT[AFD] all-fall-down signal is asserted, when the latch 603 is initiallyloaded, it increments the value of the MSG STAT [LENGTH] signals beforecoupling them to the latch 603. It will be appreciated that, if the leaf21 is an intermediate destination for a data router message packet 30while the data router 15 is in all-fall-down mode, the length of thepacket 30 is actually the length as represented by the contents of themessage length field 34, which, in turn, is represented by thebinary-encoded value of the [LEFT] MSG STAT [LENGTH] signals,incremented by one. This accommodates the additional word constitutingthe header 40, down path identification portion 41, and the messagelength and tag fields 34 and 35, which are retained by the port ejector225(l).

When the [LEFT] MSG STAT message status signals are loaded into thelatches 601, 604 and 605, they are also loaded into a shadow register610. In particular, the [EFT] MSG STAT [AFD] signal is loaded into alatch 611(l), the [LEFT] MSG STAT [LENGTH] signals are loaded into alatch 612(l), and the [LEFT] MSG STAT [TAG] signals are loaded into alatch 613(l). In addition, a valid flag 614(l) is conditioned toindicate the status of the contents of the latches 611(l) through 613(l)in the shadow register 610(l). The valid flag 614(l), when set, assertsa [LEFT] VALID signal. A shadow register 610(r) is provided, havingsimilar latches 611(r) through 613(r) and a similar valid flag 614(r)for the [RIGHT] MSG STATUS signals from the right ejector port 225(r).When the latches 601, 604 and 605 are loaded from the interface bus 211,the corresponding latches of the shadow register 610(l) are also loadedtherefrom.

The latched signals output from both shadow registers 610(l) and 610(r)are coupled to respective data input terminal of a multiplexer 614,whose data output terminal is connected to latches providing thereceived length field 243 and the received tag field 245 of the statusregister 231, the received all-fall-down field 254 of the privateregister 22, and to a selection and decrementation circuit 615. Theselection and decrementation circuit 615, like circuit 602, controlsselection and decrementation of signals coupled to a latch 244, whichprovides the length remaining field 244 of the status register 231. Thelatches 243, 244, 245 and 254 are loaded in response to a [DR] NEWSTATUS signal, which are generated in response to the [LEFT] NEW STATUSand [LEFT] VALID signals, as well as the [RIGHT] NEW STATUS and [RIGHT]VALID signals.

The multiplexer 614 is controlled by an EJ PORT PTR [LEFT] ejector portpointer [left] signal generated by control circuitry described below inconnection with FIG. 9D-6. When the EJ PORT PTR [LEFT] signal isasserted, if the processor 200 reads the middle receive register 260(FIG. 9A-2A) the left ejector port 225(l) provides the data. On theother hand, when the EJ PORT PTR [EFT] signal is negated, if theprocessor 200 reads the middle receive register 260 (FIG. 9A-2A) theright ejector port 225(r) provides the data. Accordingly, if the EJ PORTPTR [LEFT] signal is asserted, the multiplexer 614 will couple thecontents of the left shadow register 610(l) to latches 243, 244, 245 and254 for loading if the processor 200 reads from the middle receiveregister 260. Otherwise, the multiplexer 614 will couple the contents ofthe right shadow register 610(r) thereto.

The contents of the latches 243, 244, 245 and 254 can also be read onto,and provided by, respective lines of the bus 211 in the same manner aslatches 601, 603, 604 and 605, as described above. The interface 212,under control of the processor 200, may provide signals, generallyidentified as PEI BUS WRT DR REG and PEI BUS RD DR REG signals, tocontrol writing to and reading from the respective latches.

FIG. 9D-2 depicts circuitry for controlling coupling of respectivesignals for storage in the latches 601, 604 and 605, and the selectionand decrementation circuit 602 for controlling coupling of signals forstorage in the latch 603. With reference to FIG. 9D-2, the [LEFT] MSGSTAT [AFD] signal, from the left ejector port 225(l), is coupled to onedata input terminal of a multiplexer 620(l). The PEI BUS [AFD] signal isalso coupled to the other data input terminal of multiplexer 620(l). Ifthe [LEFT] NEW STATUS signal is asserted, the multiplexer 620(l) couplesthe [LEFT] MSG STAT [AFD] signal from the status information FIFO 560 asa [LEFT] SEL [AFD] left selected all-fall-down signal, to the data inputterminal of latch 605. the assertion of the [LEFT] NEW STATUS signalalso energizes and OR gate 621(l) to assert a [LEFT] LD [AFD] left loadall-fall-down signal, which clocks the latch 605, enabling it to latchthe [LEFT] SEL [AFD] signal.

On the other hand, if the [LEFT] NEW STATUS signal is negated, themultiplexer 620(l) couples the PEI BUS [AFD] signal to the data inputterminal of latch 605. If a PEI BUS WRT LDR PRVT REG write left datarouter private register signal is asserted, the OR gate 621(l) is alsoenergized to assert the [LEFT] LD [AFD] signal.

Similarly, the latches 604 and 601 are associated with multiplexers622(l) and 623(l), respectively. Multiplexer 622(l) receives, at itsdata input terminals, the PEI BUS [TAG] and the [LEFT] MSG STAT [TAG]signals, and, under control of the [LEFT] NEW STATUS signal, couples oneof them as the [LEFT] SEL [TAG] selected tag signals to the data inputterminal of latch 604. Multiplexer 623(l) receives at its data inputterminals, the PEI BUS [LEN] and the [LEFT] MSG STAT [LEN] signals, and,under control Of the [LEFT] NEW STATUS signal, couples one of them asthe [LEFT] SEL [LEN] selected tag signals to the data input terminal oflatch 601. The assertion of the [LEFT] NEW STATUS signal also energizesan OR gate 624(l) to assert a [LEFT] LD [TAG/LEN] left load tag andlength signal, which enables the latches 604 to latch the respective[LEFT] SEL [TAG] and [LEFT] SEL [LEN] signals. If the [LEFT] NEW STATUSsignal is negated, the multiplexers 622(l) and 623(l) couple the PEI BUS[TAG] and PEI BUS [LEN] signals to the data input terminals of latches604 and 601. If a PEI BUS WRT LDR STA REG write left data router statusregister signal is asserted, the OR gate 624(l) is also energized toassert the [LEFT] LD [TAG/LEN] signal.

As noted above, the contents of latches 601, 604 and 605 can also becoupled onto the interface bus 211. Associated with each latch is agated driver 625(l), 626(l) and 627(l), which, when energized by a PEIBUS RD LDR STA REG read left data router status register signal, or aPEI BUS RD LDR PRVT REG read left data router private register signal,couples the contents of the respective latches onto respective lines ofthe interface bus 211.

The selection and decrementation circuit 602 includes gated drivers630(l), 631(l) and 632(l) which receives signals from the three sourcesfrom which the length remaining latch 603 may be loaded. Gated driver630(l) enables the length information from the interface bus 211 to beloaded into the latch 603. Gated driver 630(l), under control of anasserted PEI BUS WRT LDR STA REG signal, gates the PEI BUS [LEN] signalsto an OR circuit 624(l). If the PEI BUS WRT LDR STA REG signal isasserted, the OR circuit 634(l) couples the PEI BUS [LEN] signal as[LEFT] SEL NXT LIEN REM left selected next length remaining signal. Theasserted PEI BUS WRT LDR STA REG signal also energizes an OR gate 635(l)to assert a [LEFT] LD [LEN REM] left load length remaining latch signal,which enables the latch 603 to latch the [LEFT] SEL NXT LEN REM signals.

The length remaining information from the left message ejector port225(l) is coupled through gated driver 631(l). The [LEFT] MSG STAT [LEN]signals are coupled to one data input terminal of a multiplexer 636(l).The signals are also coupled to an incrementation circuit 637(l), whichprovides signals to the other data input terminal of multiplexer 636(l)whose binary-encoded value is one greater than the binary-encoded valueof the [LEFT] MSG STAT [[LEN] signals. The incrementation provided bythe circuit 637(l) accommodates the additional message length of a datarouter message packet 30 received while the data router 15 is inall-fall-down mode and the leaf 21 is not the destination. Themultiplexer 636(l) is controlled by the [EFT] MSG STAT [AFD] signal, tocouple one of the signals at its input terminal to the gated driver631(l). If the [LEFT] MSG STAT [AFD] signal is negated, the multiplexer636(l) couples the [LEFT] MSG STAT [LEN] signals to the gated driver,and if the [LEFT] MSG STAT [AFD] signal is asserted it couples theincremented signals thereto.

If the [LEFT] NEW STATUS signal is asserted, it enables the gated driver631(l) to couple the signals selected by multiplexer 636(l) to ORcircuit 634(l), which, in turn, couples them as the [LEFT] SEL NXT LENREM signals to the latch 603. The asserted [LEFT] NEW STATUS signal alsoenergizes OR gate 635(l) to assert the [LEFT] LD [LEN REM] signal,enabling the latch 603 to latch the [LEFT] SEL NXT LEN REM signals.

The [LEFT] LAT [LEN REM] left latched length remaining signals fromlatch 603 are coupled to a decrementation circuit 640(l), whichgenerates DECR LEN REM decremented length remaining signals whosebinary-encoded value is one less than the binary-encoded value of the[LEFT] LAT [LEN REM] signals. The DECR LEN REM signals are coupled togated driver 632(l). If the [LEFT] NEW STATUS and PEI BUS WRT LDR STAREG signals are negated, and if a ZERO LEFT signal is also negated, thegated driver 632(l) couples the DECR LEN REM signals to the OR circuit634(l), which couples them as the [LEFT] SEL NXT LEN REM signals tolatch 603. When the PEI BUS RD LDR REC REG signal is asserted,indicating that the processor 200 is enabling the left ejector port'sreceive register 295 to be read, the OR gate 635(l) is energized toassert the [LEFT] LD [LEN REM] signal. The asserted [LEFT] LD [LEN REM]signal enables the latch 603 to latch the [LEFT] SEL NXT LEN REMsignals. Accordingly, gated driver 632(l) enables the contents of latch603 to be decremented when the processor 200 retrieves the data routermessage packet 30 from the left receive register 295.

As with the other latches 601, 604, and 605, the contents of latch 603can be retrieved under control of the processor 200. If the PEI BUS RDLDR STA REG signal is asserted, a gated driver 641(l) couples the [LEFT]LAT [LEN REM] signals onto the interface bus 211.

As noted above, gated driver 632(l) is also controlled by a ZERO LEFTsignal. The ZERO LEFT signal is generated by an AND gate 642(l), whichis energized by the coincidence of the PEI BUS RD DR STA REG and the EJPORT PTR LEFT signals. When the ZERO LEFT signal is asserted, driver632(l) is disabled, effectively negating the signals coupled to ORcircuit 634(l). The processing element interface 212 asserts the PEI BUSRD DR STA REG when the processor 200 enables retrieval of the contentsof the middle status register 231. With the coincident assertion of theEJ PORT PTR LEFT signal, the contents of the middle status register 231is provided by the left status register 293. When the processorsubsequently retrieves the data router message packet 30 by retrievalsfrom the middle receive register 233, the data router message packet 30will be provided by the left receive register 295. Since the data routermessage packet 30 whose length is represented by the contents of thelatch 603 will be retrieved through the middle receive register 233, thelatch 603 is zeroed, by the assertion of the ZERO LEFT signal, toprevent the processor 200 from also attempting to retrieve the same datarouter message packet 30 by retrieval requests directed to the leftreceive register 295.

FIG. 9D-3 depicts the left shadow register 610(l) and circuitry forcontrolling contents thereof. As with latches 605, 604 and 601 (FIG.9D-2), the data input terminals of latches 613(l), 612(l) and 611(l)receive signals from multiplexers, identified by reference numerals650(l), 651(l) and 651(l), respectively. Under control of the PEI BUSWRT LDR STA PEG signal from the processor element interface 212, themultiplexer 650(l) selectively couples either the [LEFT] MSG STAT [TAG]signals or the PEI BUS [TAG] signals as [LEFT] SEL SHAD TAG leftselected shadow tag signals to the data input terminal of the latch613(l) of shadow register 610(l). Similarly, the multiplexer 651(l)selectively couples either the [LEFT] MSG STAT [LEN] signals or the PEIBUS [LEN] signals to the dam input terminal of the latch 612(l) ofshadow register 610(l). If either the PEI BUS WRT LDR STA REG signal orthe [LEFT] NEW STATUS signal is asserted, an OR gate 653 is energized toassert a [LEFT] LD SHAD STA REG left load shadow status register signal,which enables the respective latches 612(l) and 613(l) to latch thesignals at their data input terminals.

Similarly, under control of the PEI BUS WRT LDR PRVT REG signal from theprocessor element interface 212, the multiplexer 650(l) selectivelycouples either the [LEFT] MSG STAT [AFD] signal or the PEI BUS [AFD]signal as [LEFT] SEL SHAD AFD left selected shadow all-fall-down signalto the data input terminal of the latch 611(l) of shadow register610(l). If either the PEI BUS WRT LDR PRVT REG signal or the [LEFT] NEWSTATUS signal is asserted, an OR gate 654 is energized to assert a[LEFT] LD SHAD PRVT REG left load shadow private register signal, whichenables latch 611(l) to latch the signals at its data input terminal.

FIG. 9D-3 also depicts circuitry for controlling left valid flag 614(l),which generates the LEFT VALID signal. In particular, a multiplexer 660receives a [LEFT] NXT VAL left next valid signal from an OR gate 661 anda WRT LDR STA VALID write left status register/valid signal from amultiplexer 662. The OR gate 661 asserts the [LEFT] NXT VAL signal undertwo general circumstances. In particular, if, when the left messageejector port 225(l) is asserting the [LEFT] NEW STATUS signal indicatingreceipt of a data router message packet 30 to be retrieved by theprocessor 200, the [LEFT] VALID signal is negated by a reset valid flag614(l), a VAL NEW STAT valid because of new status signal is asserted.The asserted VAL NEW STAT signal energizes OR gate 661 to assert the[LEFT] NXT VAL signal.

In addition, if, while the EJ PORT PTR LEFT signal is asserted, theprocessor 200 initiates a retrieval operation through the middle receiveregister 233, which enables the processor element interface 212 toassert the PEI BUS RD DR REC KEG read middle receive register signal, anAND gate 664 is energized to assert a DR READ FRM LEFT signal. Theassertion of either the DR READ FRM LEFT signal or the PEI BUS RD LDRREC REG, which indicates that the processor 200 is initiating aretrieval operation through the left receive register 295, energizes theOR gate 665 to assert a LEFT RD signal. If the [LEFT] VALID signal isthen negated, an AND gate 666 is energized to assert a VAL RD LEFT validon read from left signal, which also energized OR gate 661 to assert the[LEFT] NXT VAL left next valid signal. If the processor elementinterface 212 is not asserting the PEI BUS WRT LDR STA REG, which occurswhen the processor 200 is enabling the left status register 293 to beloaded, the multiplexer 660 couples the [LEFT] NXT VAL left next validsignal to the data input terminal of valid flag 614(l), to condition theflag 614(l).

On the other hand, if the PEI BUS WRT LDR STA KEG signal is asserted,indicating that the processor 200 is enabling loading of the left statusregister 293, the multiplexer 660 couples a WRT LDR STA VALID write leftstatus register valid signal to condition valid flag 614(l). The WRT LDRSTA VALID signal is provided by multiplexer 662. In particular, if theall-fall-down latch 605 (FIG. 9D-2) is negating the [LEFT] LAT [AFD]signal, multiplexer 662 couples an output signal from a comparator 670,which generates an asserted signal if the PEI BUS [LEN] signals and thePEI BUS [LEN REM] signals have the same binary-encoded value, as the WKTLDR STA VALID signal. On the other hand, if the all-fall-down latch 605is asserting the [LEFT] LAT [AFD] signal, multiplexer 662 couples, asthe WRT LDR STA VALID signals, an output signal from a comparator 671,which generates an asserted signal if the PEI BUS [LEN REM] signals havethe same binary-encoded value as that of the PEI BUS [LEN] signals, asincremented by one by an incrementation circuit 672.

The comparators 670 and 671 enable conditioning of the valid flag 614(l)using the contents of the receive length and receive length remainingfields of the left status register 293 as it is being loaded by theprocessor 200, which will occur, for example, during a context switchoperation. In particular, if the length (as adjusted by theincrementation circuit 672 if the [LEFT] LAT [AFD] signal is asserted)and length remaining values are the same, no part of the data routermessage packet 30 whose status is indicated by the length and lengthremaining fields in the status register will not have been retrieved,and so the processor 200 could enable retrieval of the message packet 30through either the left receive register 295 or the middle receiveregister 233.

On the other hand, if the lengths indicated by the values of the length(as adjusted) and length remaining fields differ, the processor 200 willhave partially retrieved the data router message packet 30 through theleft receive register 295, and it will resume retrieval therethroughafter the context has been switched. In that case, the WRT LDR STA VALIDsignal will be negated, which will enable the multiplexer 660 to, inturn, enable the valid flag 614(l) to be clear.

The [LEFT] VALID and [RIGHT] VALID signals are used by the circuitry forcontrolling the middle status and private registers 231 and 232, whichis depicted in FIGS. 9D-4 and 9D-5. Much of the circuitry depicted onFIGS. 9D-4 and 9D-5 is similar to that described above in connectionwith FIG. 9D-3, and will not be described herein. In particular,multiplexers and gates for controlling the loading of the respectivelatches 254, 245, 243 and 244, where similar, is identified by referencenumerals having the tag "(M)" instead of "(l)", and such circuitryoperates in substantially the same way. FIG. 9D-4 also includescircuitry for generating a [MID] NEW STATUS signal. In addition, thecircuitry depicted on FIGS. 9D-4 and 9D-5 includes multiplexers 680through 684 which, under control of the EJ PORT PTR LEFT signal,selecting between the left and right shadow registers 610(l) and 610(r)for providing signals to the respective multiplexers 620(M), 622(M),623(m) and 636(m), substituting for the [LEFT] MSG STAT signals in thecircuit depicted in FIG. 9D-2. The EJ PORT PTR LEFT signal, whenasserted, enables retrievals initiated by the processor 200 from themiddle receive register 233 to be provided by the left message ejectorport 225(l). On the other hand, when the EJ PORT PTR LEFT signal isnegated, the retrievals initiated by the processor from the middlereceive register 233 are provided by the right message ejector port225(r).

The [MID] NEW STATUS signal is generated by a multiplexer 690 and ANDgates 691 and 692. If the [LEFT] NEW STATUS signal is asserted, and ifthe valid flag 614(l) is asserting the [LEFT] VALID signal, AND gate 691is energized to assert a [MID] NEW [LEFT] middle new status from leftsignal. If the EJ PORT PTR LEFT ejector port pointer left signal isasserted the multiplexer 690 couples the [MID] NEW [LEFT] signal as the[MID] NEW STATUS signal. If either the [LEFT] NEW STATUS or the [LEFT]VALID signals are negated when the EJ PORT PTR LEFT signal is asserted,the [MID] NEW STATUS signal will be negated.

Similarly, if the [RIGHT] NEW STATUS signal is asserted, and if thevalid flag 614(R) is asserting the [RIGHT] VALID signal, AND gate 692 isenergized to assert a [MID] NEW [RT] middle new status from rightsignal. If the EJ PORT PTR LEFT ejector port pointer left signal isnegated, to identify the left message ejector port 225(r), themultiplexer 690 couples the [MID] NEW [RT] signal as the [MID] NEWSTATUS signal. If either the [RIGHT] NEW STATUS or the [RIGHT] VALIDsignals are negated when the EJ PORT PTR LEFT signal is negated, the[MID] NEW STATUS signal will also be negated.

FIG. 9D-6 depicts circuitry for controlling the condition of the EJ PORTPTR LEFT signal. Generally, the EJ PORT PTR LEFT signal is controlled inresponse to the receipt by the left and right message ejector ports225(l) and 225(r) of a data router message packet 30 to be retrieved bythe processor 200, by the processor 200 enabling retrieval of a datarouter message packet 30 through either the left or right receiveregisters 295 and 303, and by the condition of the EJ PORT PTR LEFTsignal. In particular, if, for example, the left message ejector port225(l) has a new data router message packet 30 for retrieval by theprocessor 200, and the right message ejector port 225(r) does not, andif further the EJ PORT PTR LEFT signal is asserted, the circuit enablesthe EJ PORT PTR LEFT to remain asserted. On the other hand, if bothmessage ejector ports 225(l) and 225(r) have new data router messagepackets 30 for retrieval by the processor 200, the EJ PORT PTR LEFT isenabled to toggle.

Retrieval requests from the processor 200 also can change the conditionof the EJ PORT PTR LEFT signal. In particular, if the processor enablesa retrieval operation from the left receive register 295, while the EJPORT PTR LEFT signal is asserted, the circuit negates the EJ PORT PTRLEFT signal. On the other hand, if the processor 200 enables a retrievaloperation from the right receive register 302 while the EJ PORT PTR LEFTsignal is negated, the circuit asserts the EJ PORT PTR LEFT signal.

More specifically, with reference to FIG. 9D-6, if at least one of the[LEFT] MSG STAT [LIEN] signals is asserted, an OR gate 700(l) isenergized to assert a RCV MSG LEFT received message left signal. Thiswill occur if the [LEFT] MSG STAT [LEN] signals from the left messageejector port 225(l) indicate receipt of a new data router message packet30. If an OR gate 700(r), which receives the [RIGHT] MSG STAT [LEN]signals, generates a negated RCV MSG RT signal, indicating that theright message ejector port 225(r) does not have a new data routermessage packet 30, an AND gate 701(l) is energized, to assert a PT LEFTREQ point left request. If the EJ PORT PTR LEFT signal is asserted, andif a periodically-asserted UPDATE PTR EN update pointer enable signal isalso asserted, an AND gate 702(l) is energized to assert a TO LEFTsignal. The TO LEFT signal, in turn, energizes an OR gate 703 to assertan EN NEXT enable next signal which is coupled to the data inputterminal of a flip-flop 707. The asserted UPDATE PTR EN signal alsoenables an OR gate 708 to assert an UPDATE FTR update pointer signal,which enables the flip-flop 707 to be set in response to the asserted ENNEXT signal, thereby asserting the EJ PORT PTR LEFT signal.

Similarly, if at least one the [RIGHT] MSG STA [LEN] signal is asserted,and the [LEFT] MSG STAT [LEN] signals are negated, an AND gate 701(R)asserts a PT RT REQ point right request. If the EJ PORT PTR LEFT signalis negated, AND gate 702(r) will be energized to assert a TO RIGHTsignal, which disables OR gate 703 to negate the EN NEXT signal. Whenthe UPDATE PTR signal is asserted, the flip-flop 707 will be reset tonegate the EJ PORT PTR LEFT signal.

On the other hand, if both RCV MSG LEFT and RCV MSG RT signals areasserted, indicating both left and right message ejector ports 225(l)and 225(r) have received data router message packets 30 for retrieval bythe processor 200, an AND gate 704 asserts a BOTH REQ both requestsignal, which is coupled to one input terminal of an XOR exclusive ORgate 705. If the EJ PORT PTR LEFT signal is asserted, the XOR gate 705negates a CH PTR REQ change pointer request signal, which disables anAND gate 706 to negate a TO NEXT signal. The negated TO NEXT signaldisables OR gate 703 to negate the EN NEXT signal, which, in turn,enables the flip-flop 707 to be reset when the UPDATE PTR update pointersignal is next asserted. On the other hand, if the EJ PORT PTR LEEsignal is negated when the BOTH REQ both requested signal is asserted,AND gate 706 is energized to assert the TO NEXT signal, which enables ORgate 703. Accordingly, flip-flop 707 will be energized in response tothe assertion of the UPDATE PTR signal.

As noted above, the EJ PORT PTR LEFT signal can be controlled inresponse to requests from the processor 200 initiating retrievals fromthe left and right receive registers 295 and 304. In particular, if theprocessing element interface 212 is asserting the PEI BUS RD LDR REC REGsignal, indicating that the processor 200 is requesting a retrieval fromthe left receive register 295, and if the EJ PORT PTR LEFT signal isasserted, an AND gate 710 is energized to assert a TO RIGHT/FROM LDR RECREQ signal, which is complemented to disable the OR gate 703. Thedisabled OR gate 703 provides a negated EN NEXT signal, which resets theflip-flop 707 in response to the assertion of the UPDATE PTR signal,thereby negating the EJ PORT PTR LEFT signal.

On the other hand, if the processing element interface 212 is assertingthe PEI BUS RD RDR REC REG signal, indicating that the processor 200 isrequesting a retrieval from the left receive register 295, and if the EJPORT PTR LEFT signal is negated, an AND gate 711 is energized to asserta TO LEFT/FROM RDR REC REQ signal, which is complemented to enable theOR gate 703. The enabled OR gate 703 provides an asserted ENNEXT-signal, which sets the flip-flop 707 in response to the assertionof the UPDATE PTR signal, thereby asserting the EJ PORT PTR LEFT signal.

FIG. 9D-7 depicts the message count register 313 (FIG. 9A-2B) andcircuitry for controlling the register 313. As noted above, and as willbe described in detail below, the contents of the message count register313 are incremented in response to the transmission by the leaf 21 ofdata router message packets 30, and decremented in response to thereceipt thereof. In addition, CUR MSG CNT current message count signals,which represent the value contained in the message count register 313,are used in control network message packets 60 to determine when thedata router 15 is empty.

With reference to FIG. 9D-7, the message count register 313 may bewritten with an initial value by the processor 200 through a multiplexer720. In that operation, the processor 200 provides the initial value tothe processor element interface 212, and enables it to transfer thevalue as PEI BUS [MSG CNT] message count signals and assert a PEI BUSWRT MSG CNT REG write message count register enable signal. In responseto the PEI BUS WRT MSG CNT REG signal, the multiplexer 720 couples thePEI BUS [MSG CNT] signals as NXT MSG CNT next message count signals todata input terminals of the message count register. The asserted PEI BUSWRT MSG CNT REG signal also energizes an OR gate 721 to assert a MSG CNTLD message count load signal, enabling the message count register 313 toload the NXT MSG CNT signals.

Incrementation and decrementation of the message count value stored inthe message count register 313 are accomplished by an incrementationcircuit 722 and a decrementation circuit 723, respectfully, undercontrol of a control circuit 724. The control circuit 724 generates anINC MSG CNT EN increment message count enable signal to enable themessage count value in the register 313 to be incremented, and a DEC MSGCNT EN decrement message count enable signal. Generally, the controlcircuit 724 asserts the INC MSG CNT EN signal in response totransmission of a data router message packet 30 by the left and rightmessage injector ports 223(l) and 223(r). In addition, the controlcircuit asserts the DEC MSG CNT EN signal in response to reception of adata router message packet 30 by the left and right message ejectorports 225(l) and 225(r).

The incrementation circuit 722 receives the CUR MSG CNT current messagecount signals from the message count register 313 and generates INC MSGCNT incremented message count signals in response. The INC MSG CNTsignals have a binary-encoded value that is one greater than thebinary-encoded value of the CUR MSG CNT signals. Similarly, thedecrementation circuit 723 receives the CUR MSG CNT signals andgenerates DEC MSG CNT decremented message count signals having abinary-encoded value that is one less than the binary-encoded value ofthe CUR MSG CNT signals.

Both the INC MSG CNT signals and the DEC MSG CNT signals are coupled todata input terminals of a multiplexer 725, which is controlled by theINC MSG CNT EN increment message count enable signal from the controlcircuit 724. If the INC MSG CNT EN signal is asserted, the multiplexer725 couples the INC MSG CNT signals as ADJ MSG CNT adjusted messagecount signals to the second data input terminals of multiplexer 720.Similarly, if the control circuit 724 is not asserting the INC MSG CNTEN signal, the multiplexer 725 couples the DEC MSG CNT signals fromdecrementation circuit 723 as the ADJ MSG CNT signals. In either case,at this point the PEI BUS WRT MSG CNT REG is negated, which enables themultiplexer 720 to couple the ADJ MSG CNT signals as the NXT MSG CNTsignals to the message count register 313.

If the control circuit 724 is not also at that point asserting the DECMSG CNT EN decrement message count enable signal, an XOR exclusive ORgate 726 is energized to assert a CH MSG CNT change message countsignal. The asserted CH MSG CNT signal energizes OR gate 721 to assertthe MSG CNT LD signal, to enable the message count register 313 to loadthe NXT MSG CNT signals from multiplexer 720. It will be appreciatedthat, if the control circuit 724 is asserting both the INC MSG CNT ENsignal and the DEC MSG CNT EN signal, the XOR gate 726 will bede-energized to negate the CH MSG CNT signal. If the PEI BUS WRT MSG CNTREG signal is also negated, OR gate 721 will be de-energized, negatingthe MSG CNT LD signal and disabling the message count register 313 fromloading the NXT MSG CNT signals. The XOR gate 726 is provided since, ifcontrol circuit 724 is simultaneously asserting both the INC MSG CNT ENsignal, to enable incrementation of the value in 313 message countregister, and the DEC MSG CNT EN signal, to enable decrementation of thevalue in the message count register 313, the value in the register 313will remain unchanged.

The control circuit 724 includes an incrementation enabling portion 730,which controls the INC MSG CNT EN signal, and a decrementation enablingportion 731, which controls the DEC MSG CNT EN signal. Theincrementation enabling portion 730, generally, controls the INC MSG CNTEN signal in response to XMIT MSG STAT [TAG] (x) signals ("x"referencing "L" or "R") representative of the message tag field of thedata router message packet 30 to be transmitted, the [xDR] MSG AVAILsignal from the left or right message injector port 223(l) or 223(r)indicating that the packet is being transmitted, and COUNT MASK signalsprovided by a count mask register (not shown).

As with other registers, the count mask register may be loaded by theprocessor 200 The COUNT MASK signals permit the message count value inthe register 313 to be selectively adjusted based on the XMIT MSG STAT[TAG] signals of the message packet 30 being transmitted or received,and so the processor 200 can control particular conditions, representedby the various encodings of the tag fields 35, under which transmissionor reception of messages will affect the message count value in themessage count register 313. In the incrementation enabling portion 730,the COUNT MASK signals are directed to data input terminals ofmultiplexers 732(x), which is controlled by the XMIT MSG STAT [TAG] (x).

In response, each multiplexer 732(x) couples one of the COUNT MASKsignals, selected based on the encoding of the XMIT MSG STAT [TAG] (x)signals, as an EN INC (x) enable increment signal. Whether the EN INC(x) enable increment signal is asserted will be determined in responseto the particular assertion pattern of the COUNT MASK signals and theencoding of the XMIT MSG STAT [TAG] (x) signals. The SEL CNT (x) signaland xDR MSG AVAIL signal ("x" referencing "L" and "R") are coupled to anAND gate 733(x). If both signals are asserted, the AND gate 733(x) isenergized to assert an INC CNT (x) increment count signal, whichenergizes and OR gate 734 to assert the INC MSG CNT EN increment messagecount enable signal. On the other hand, if either the EN INC (x) signal,or the xDR MSG AVAIL signal are negated, AND gate 733(x) will bede-energized to negate the INC CNT (x) signal. If both the AND gates733(l) and 733(r)-are negating their INC CNT (L) and the INC CNT (R)signals, the OR gate 734 will be de-energized to negate the INC MSG CNTEN signal.

The decrementation enabling portion 731 has similar multiplexers 740(x)and AND gates 741(x) and an OR gate 742 for generating the DEC MSG CNTEN decrement message count enable signal. The multiplexers 740(x) arecontrolled by the [LEFT] MSG STAT [TAG] and [RIGHT] MSG STAT [TAG]signals from the left and right message ejector portions 225(l) and225(r) respectively. In response to the [x] MSG START [TAG] signals, thecorresponding multiplexer 740(x) couples a selected one of the COUNTMASK signals as an EN DEC (x) enable decrement signal to the AND gate741(x). AND gate 741(l) is also controlled by the [LEFT] NEW STATUSsignal from left message ejector port 225(l). If the EN DEC (L) and[LEFT] NEW STATUS signals are asserted, AND gate 741(l) is energized toassert a DEC CNT (L) decrement count signal, which energizes OR gate 742to assert the DEC MSG CNT EN signal.

The AND gate 741(r), on the other hand, is controlled by a GATED [RT]NEW STATUS signal from a buffer circuit 750. Buffer circuit 750 buffersand delays the effect of the assertion of the [RIGHT] NEW STATUS signalin connection with AND gate 741(r) if both it and the [LEFT] NEW STATUSsignal are asserted simultaneously, indicating simultaneous reception ofdata router message packets 30 by both message ejector ports 225(l) and225(r). If the [RIGHT] NEW STATUS signal is asserted, but the [LEFT] NEWSTATUS signal is negated, an AND gate 751 is energized, which, in turn,energizes an OR gate 752 to assert the GATED [RT] NEW STATUS signal.

On the other hand, if both the [RIGHT] NEW STATUS and the [LEFT] NEWSTATUS signals are asserted, AND gate 751 is de-energized, and an ANDgate 753 is energized. The energized AND gate 753 provides an assertedsignal to the data input terminal of a flip-flop 754, which is set atthe next tick of the NODE CLK signal to assert a BUF [RT] NEW STATUSbuffered right new status signal. The asserted BUF [RT] NEW STATUSsignal energizes OR gate 752 to assert the GATED [RT] NEW STATUS signal,which enables the AND gate 741(r). Thus, if both the [LEFT] NEW STATUSand the [RIGHT] NEW STATUS signals are asserted simultaneously, buffercircuit 750 delays the enablement of the AND gate 741(r), which controlsthe DEC CNT (R) signal, to ensure decrementation of the value in themessage count register 313 in response to reception of data routermessage packets 30 by both message ejector ports 225(l) and 225(r).

iii. Control Network Interface 204

a. General

As noted above, the control network interface 204 receives (ejects)control network message packets 60 from the control network 14, andtransmits (injects) control network message packets 60 to the controlnetwork 14. A general block diagram of control network interface 204 isshown in FIG. 10A, and more detailed diagrams are shown in FIGS. 10Bthrough 10G.

With reference to FIG. 10A, the control network interface 204 includes atransmit section 800 that transmits control network message packets 60over the control network 14, a receive section 801 that receives controlnetwork message packets 60 from the control network 14, and a rootcontrol/loopback section 802, all of which are controlled by a commoncontrol section 803 and set of registers 804. The transmit section 800transmits, that is, injects, control network message packets 60 over thecontrol network 14. The receive section 801 receives, that is, ejects,control network message packets from the control network 14, inparticular, from the control network node 50(1,j) (FIG. 4A) connectedthereto.

The root control/loopback section 802 determines whether the controlnetwork interface 204 is a logical root of a sub-tree, and if so assertsa ROOT UP signal, which is received by the control network node 50(1,j).It will be appreciated that, if the control network interface 204 isasserting the ROOT UP signal, it is the root of a sub-tree thatcomprises only the single leaf 21.

The common control section 803 maintains several control and statusregisters and effectively controls the operations of the interface 204as will be described below. The registers 804 contain information whichis used in generating control network message packets 60. The registers804 can be written by the processor 200 to transmit some controlinformation over the control network 14 in message packets 60transmitted by the transmit section 800. Alternatively, the registers804 may be loaded with control information which the receive section 801obtained from control network message packets 60 received thereby.

Like the data router interface 205, the control network interface 204also makes use of a number of registers to enable the processor 200 toinitiate transfers of message packets 60 over the control network 14,and facilitate reception by the processor 200 of data from controlnetwork message packets 60 that the control network interface 204receives from the control network 14. In particular, the processor mayinitiate transmissions over the control network 14, by loadinginformation in a supervisor broadcast register set, a broadcast registerset and a combine register set provided in the control network interface204. All of the register sets are generally similar to the send first,send and receive registers 234, 235, and 233, respectively of the datarouter interface 205 (FIG. 9A-2A), except that the first word of thesend first register in the control network interface includes diversefields, as will be described in connection with FIG. 10A-2. Since thesend and receive registers in each of the supervisor broadcast,supervisor, and combine register sets are similar to thecorrespondingly-named registers as shown on FIG. 9A-2A, they will not bedescribed further herein.

The processor 200 enables the control network interface 204 to generatecontrol network message packets 60 in a manner similar to that in whichit enables the data router interface 205 to generate data router messagepackets 30. In particular, the processor 200 first transmits informationto the control network send first register. Thereafter, if the controlnetwork message packet 60 is, for example, for a multi-word scanoperation, requiring multiple control network message packets 60 eachhaving a thirty-two bit word, the processor 200 can provide theadditional words by loading them into the send register in theparticular register set.

When the control network interface 204 receives a control networkmessage packet 60 from the control network 14, it enables the processingelement interface 212 to interrupt the processor 200, identifying theparticular register set into which the information from the messagepacket 60 was loaded. The processor 200 can obtain the data receivedfrom a particular control network message packet 60 by retrieving thecontents of the receive register in the particular register setidentified in the interrupt.

The send first registers in the supervisor broadcast, broadcast andcombine register sets all have the same general structure, which isshown in FIG. 10A-2. With reference to FIG. 10A-2, each send firstregister includes two words 806 and 807. A high-order address field 808in the first word 806 contains an address value, in the address spacedefined for the memory bus 203 (FIG. 8) of the network interface 202and, specifically, of the control network interface 204.

The remaining fields 819 through 829 of the first word 806 includeinformation which the control network interface 204 uses in generatingthe packet header 61. A combine function field 819, and pattern field828 in the first word 806 of the send first register contain informationthat is copied into fields 65, 66 and 67, respectively, of thepacket-header 61. The second word 807 of a send first register containsdata to be transmitted in data fields 70 of the packet data 62. If thecontents of length field 829 if the send first register indicates that amessage is to include multiple thirty-two bit words, each thirty two bitword is sent in a control network message packet 60, with the contentsof successive data fields 70 being provided through the send register.

Returning to FIG. 10A-1, the transmit section includes three first-infirst-out buffers, namely, a supervisor broadcast transmit first-infirst-out buffer (FIFO) 810, a broadcast transmit FIFO 811 and a combinetransmit FIFO 812. Generally, the supervisor broadcast transmit FIFO 810stores information used by the control network interface 204 in creatingcontrol network message packets 60 of the single source message type,while the processor 200 is in its supervisor operating mode. Similarly,the broadcast transmit FIFO 811 stores information used by the controlnetwork interface 204 in creating control network message packets 60 ofthe single-source message type, while the processor 200 is in its useroperating mode. The combine transmit FIFO 812 stores information used bythe control network interface in creating control network messagepackets 60 of the multiple-source message type, including router donepackets 60 which are used to determine if the data router 15 is empty.Information in the combine transmit FIFO 812 that enables creation ofpackets other than a router done packet includes the data that will beused in establishing the contents of the data nibbles 70(i) (FIG. 5) ofthe packet 60. However, information in the combine transmit FIFO 812that enables creation of router done packets does not include such data;the data is instead provided by the CUR MSG CNT current message countsignals from the data router interface 205.

Each FIFO 810 through 812 has data input terminals that are connected tothe interface bus 211 and is loaded by the interface 212 with thecontents of the send first and send registers of the respectivesupervisor broadcast, broadcast and combine register set when theprocessor 200 transfers information thereto. The interface 212 asserts aPUSH XMIT SBC FIFO push transmit supervisor broadcast first-in first-outbuffer signal, a PUSH XMIT BC FIFO push transmit broadcast first-infirst-out buffer signal or a PUSH XMIT COM FIFO push transmit combinefirst-in first-out buffer signal to enable the respective FIFO 810through 812 to receive and store the information.

Each FIFO 810 through 812 generates status signals "XXX" XMIT FIFO FULLtransmit buffer full signal and "XXX" XMIT FIFO MT transmit buffer emptysignal ("XXX" may comprise "SBC" which identifies the supervisorbroadcast transmit FIFO 810, "BC" which identifies the broadcasttransmit FIFO 811, or "COM" which identifies the combine transmit FIFO812) which indicate whether the respective buffer is nearly full ornearly empty. If a particular FIFO 810 through 812 is asserting its"XXX" XMIT FIFO FULL signal, the interface 212 rejects attempts by theprocessor 200 to load information therein.

Each FIFO 810 through 812 also includes data output terminals that areconnected to a transmit message select and assembler circuit 813. Undercontrol of a LOAD NEW MSG load new message signal, circuit 813 receivesthe "XXX" XMIT FIFO MT signals, determines whether any of them haveinformation to be used in a control network message packet 60, and if soassembles a control network message packet 60. In assembling the messagepacket, the circuit 813 may also use the contents of registers 804 andCUR MSG CNT current message count signals from the data router interface205 if the PKT TYPE XMIT signals indicate that the control networkmessage packet 60 is a router done packet.

The transmit message select and assembler 813 couples forty-eight bitwords representing the header 61 and data portion 62 of the assembledmessage packet 60 to a transmit message buffer 814, which latches it inresponse to the LOAD NEW MSG signal. In response to successive ticks ofthe NODE CLK signal, a flick generator iteratively selects four-bitnibbles from the transmit message buffer 814 and appends to eachselected nibble a high-order bit comprising the tag bit. The flickgenerator 815 transmits the result as FLICK OUT (4:0) flick outputsignals to the control network node 50(1,j) connected thereto, and alsoto a flick buffer 816 in the root control/loopback circuit 802.

As it iteratively transmits the FLICK OUT (4:0) signals, the flickgenerator 815 generates a checksum, which it transmits as the thirteenthflick of the control network message packet. Contemporaneously withtransmission of the last flick of the message packet 60, the flickgenerator 815 asserts the LOAD NEW MSG load new message signal to enablethe transmit message buffer 814 to latch a new word and the circuit 813to generate a new control network message packet 60 for transmission.

As noted above, the control network interface 204 includes a set ofregisters 804 that provide information which may also be used ingenerating message packets. A globals register 820 contains global bitsthat can be used to perform a global operation as described above. Aflush flag 821 can be used to control the flush bit 75 in the controlnetwork message packet 60; if set, the flush bit 75 enables the controlnetwork 14 to flush intermediate results of a scan operation. Aninterrupt register 822 can be loaded with an interrupt value that can betransmitted in a single-source message packet of the interrupt type, tobroadcast interrupt information to other leaves 21 in the partition.

A configuration register 823 contains a value that can be used in asingle-source message packet of the configuration type to identifies thelevel and sub-level at which the logical root is to be established forthe partition; this information is loaded into flicks 70(0) and 70(1) ofthe packet data portion 62. An all-fall-down mode flag 824, which isderived from all-fall-down enable bit 256 of the private register 232(FIG. 9A-2A) is used to initiate an all-fall-down operation in the datarouter 15; the all-fall-down mode flag 824 is used to conditionall-fall-down mode bit 81 of the control network message packet 81.Finally, a segment flag 825, which may be conditioned by the processor200, is used in segment bit 77 of a multi-source control network messagepacket 60 to identify the beginning of a segment in a segmented scanoperation.

The receive section 801 includes a flick demultiplexer 830 thatiteratively receives, at each tick of the NODE CLK signal, either theFLICK IN signals from the control network node 50(i,j) or the BUF FLICKOUT buffered flick out signals from the root control/loopback circuit802. If the root control/loopback circuit 802 is asserting SEL XMITselect transmit section signal, generally indicating that the controlnetwork interface 204 is a logical root, the flick demultiplexer 830selects the BUF FLICK OUT signals, and otherwise it selects the FLICK INsignals. The flick demultiplexer 830 strips off the tag signals, some ofwhich it buffers, and demultiplexes the other received signals to sothat successively received signals are used to form successive nibblesof a forty-eight bit word. At the same time, the flick demultiplexer 830maintains a running checksum of the signals received at each tick of theNODE CLK signal. The flick generator uses the checksum to determinewhether the control network message packet was correctly received, and,if so, it asserts a LOAD RCVD MSG load received message packet signal.

The assertion of the LOAD RCVD MSG signal enables a received messagebuffer 831 to latch the word generated by the flick demultiplexer. Inaddition, the asserted LOAD RCVD MSG signal enables a receive messagebuffer and destination select circuit 832 to examine the word containedin the received message buffer 831, and determine which of a supervisorreceiver broadcast FIFO 833, a broadcast receiver FIFO 834, a combinereceiver FIFO 835, or one of the registers 804, in which the word shouldbe loaded.

Each FIFO 833, 834 and 835 generates an "XXX" RCV FIFO NR FULL receiveFIFO nearly full signal ("XXX" may comprise "SBC" which identifies thesupervisor receiver FIFO 833, "BC" which identifies the broadcastreceiver FIFO 834, or "COM" which identifies the combine receiver FIFO835) which indicate whether the respective FIFO is nearly full. As willbe described below in connection with FIG. 10C, the "XXX RCV FIFO NRFULL signal is used by the transmit message select and assembler 813 ingenerating several of the tag signals for the various flicks. Inaddition, the flick demultiplexer 830 couples several of the tag signalswhich it receives to the flick generator to control transmission ofcontrol network message packets 60 thereby.

b. Transmit Section 800

With this background, details of various portions of the control networkinterface 204 will be described in connection with FIGS. 10B though 10G.FIG. 10B depicts a detailed functional block diagram of the transmitmessage select and assembler circuit 813 and the transmit message buffer814. With reference to FIG. 10B, the transmit message select andassembler circuit 813 includes a message priority select circuit 840that performs several functions in response to the assertion of the LOADNEW MSG load new message signal from the flick generator 815 enablegeneration of a word to be loaded into the transmit message buffer 814.

First, the message priority select circuit 840 receives status signalsfrom various sources of message information such as the various FIFOs810, 811 and 812, and the interrupt and configuration registers 822 and823. In particular, the message priority select circuit 840 receivesfrom the FIFOs 810 through 812 the "XXX" XMIT FIFO MT ("XXX" being SBC,BC and COM) supervisor broadcast, broadcast and combine transmit FIFOempty signals. If any of these signals are negated, the correspondingFIFO 810, 811 or 812 contains data to be transmitted in a controlnetwork message packet 60. In addition, the message priority selectcircuit 840 receives an UNSENT CONFIG unsent configuration signal fromthe configuration register 823 and an UNSENT INT unsent interrupt signalfrom the interrupt register indicating that the processor 200 has loadeda new value therein that is to be transmitted in a control networkmessage packet 60.

In response to the assertion of the LOAD MSG load new message signalfrom the flick generator 815, the message priority select circuit 840uses these status signals to select one FIFO as a control networkmessage packet 60 information source in response to a predeterminedpriority to use in generating a control network message packet 60. Thecircuit 840 obtains a word from the data output terminals of theselected FIFO for use in creating the control network message packet 60.In addition, the circuit 840 generates POP "XXX" FIFO ("XXX" referencingSBC, BC and COM) first-in first-out buffer pop signal, which, whenasserted, enables the respective FIFO 810, 811 and 812 to provide a newword at its data output terminals which may be used in creating asubsequent control network message packet 60. In particular, since theFIFOs 810 through 812 are first-in first-out buffers, the words providedin response to successive POP "XXX" FIFO signals will be in the order inwhich they were stored in the respective FIFO.

The circuit 840 also generates a READ INTR REG read interrupt registersignal and a READ CONFIG REG read configuration register signal, which,when asserted, enables the respective interrupt register 822 orconfiguration register 823 to couple its contents to the circuit 813.When the respective register couples its contents to the circuit 813, itwill negate its UNSENT CONFIG unsent configuration or UNSENT INT unsentinterrupt signal until it is again loaded by the processor 200.

The message priority circuit 840 also generates MSG TYPE XMIT messagetype transmit signals and PKT TYPE XMIT that are coupled to inputterminals of the transmit message buffer 814, in particular to inputterminals of nibbles 814(0) and 814(1), respectively. The signals willbe used in forming the contents of message type field 64 and packet typefield 65, respectively. It will be appreciated that the particularencoding of the MSG TYPE XMIT and PKT TYPE XMIT signals will reflect theparticular information source identified by the particular one of thePOP "XXX" FIFO signals or READ INTR REG or READ CONFIG REG signal thatit asserts.

Alternatively, it will be appreciated that all of the "XXX" XMIT FIFO MTsignals may be asserted, indicating that none of the FIFOs 810 through812 contain information to be used in establishing control networkmessage packets 60, and the UNSENT CONFIG or UNSENT INT signals may benegated when the flick generator 815 asserts the LOAD NEW MSG signal.When that occurs, the message priority select circuit 840 may generateMSG TYPE XMIT message type transmit signals that identify a controlnetwork message packet 60 of the idle type, which it couples to nibble814(0) of the transmit message buffer 814. In either case, the PKT TYPEXMIT packet type transmit signals from circuit 840 are all negated.

The transmit message select and assembler circuit 813 also includesseveral multiplexers 841, 842 and 843 which provide additional signalsthat are coupled to the transmit message buffer 814 along with the MSGTYPE XMIT and PKT TYPE XMIT signals. With reference also to FIG. 5, thePKT TYPE XMIT signals comprise only three signals that form thelow-order three bits of the flick "1." The high-order signal is obtainedfrom a multiplexer 841, which, under control of a FLICK (1) SEL flick 1select signal from message priority select circuit 840, selects either aINTR (4) signal representing the high-order bit from the interruptregister 822 or a high-order PAT(1) pattern signal from a word poppedfrom the combine first-in first-out buffer 812. If the circuit 840 isasserting a FLICK (1) OE flick 1 output enable signal, the multiplexer841 couples the selected signal to a high-order input terminal of thenibble 814(1) of transmit message buffer 814. It will be appreciatedthat the FLICK (1) SEL signal will be conditioned to select the PAT (1)signal only if the message priority select circuit 840 has selected thecombine buffer 812 as the source of information for the control networkmessage packet 60 to be transmitted. Further, the FLICK (1) OE signalwill not be asserted if the MSG TYPE XMIT message type transmit signalsfrom circuit 840 contain the encoding identifying an idle messagepacket, to ensure that the contents of the respective fields of themessage packet 60 will be zero.

Multiplexer 842 is used to select information that will be used inestablishing the low-order four bits of flick 2 of the control networkmessage packet 60. Under control of a FLICK (2) SEL flick "2" selectsignal from the circuit 840, the multiplexer 842 selects one of the INTR(3:0) interrupt signals, or COMB FTN (2:0) combine function signals anda low-order PAT (0) pattern signal. If a FLICK (2) OE flick (2) outputenable signal is asserted, the multiplexer 842 couples the selectedsignals to the input terminals of nibble 814(2) of the transmit messagebuffer 814. The INTR (3:0). interrupt signals are based on low-orderbits contained in the interrupt register 822. The COMB FTN (2:0) signalsand PAT (0) signal are from the word popped from the combine buffer 812.It will be appreciated that the FLICK (2) SEL signal will be conditionedto select the COMB FTN and PAT (0) signals only if the message priorityselect circuit 840 has selected the combine buffer 812 as the source ofinformation for the control network message packet 60 to be transmitted.Further, the FLICK (2) OE signal will not be asserted if the MSG TYPEXMIT message type transmit signals from circuit 840 contain the encodingidentifying an idle message packet, to ensure that the contents of therespective fields of the message packet 60 will be zero.

Finally, the multiplexer 843 selects information that will be used inestablishing the contents of data nibbles 70(0) through 70(7) in thecontrol network message packet 60 under control of DATA SEL data selectsignals from the message priority select circuit 840. If the PKT TYPEXMIT signals identify a router done message the DATA SEL signals willenable the multiplexer 843 to select the CUR MSG CNT current messagecount signals from the data router interface 205 to the input terminalsof nibbles 814(4) through 814(10) of the transmit message buffer 814.Otherwise, the particular source selected by multiplexer 843 in responseto the DATA SEL data select signals will correspond to the sourceidentified by the one of the POP "XXX" FIFO signals, the READ INTR REGsignal or the READ CONFIG REG signal asserted by the message priorityselect circuit 840. If the message priority select circuit 840 is alsoasserting a DATA OE data output enable signal, the multiplexer 843 willcouple the selected signals to respective input terminals of transmitmessage buffer 814. It will be appreciated that the DATA OE signal willnot be asserted if the MSG TYPE XMIT message type transmit signals fromcircuit 840 contain the encoding identifying an idle message packet, toensure that the contents of the respective fields of the message packet60 will be zero.

In addition to these signals, GLOBAL signals representing the contentsof the globals register 820 are coupled to respective input terminals ofnibble 814(11) of the transmit message buffer 814, which latches theGLOBAL signals along with the other signals at its input terminals. Inresponse to the next assertion of the LOAD NEW MSG load new messagesignal from the flick generator 815, the transmit message buffer 814latches all of the signals at its input terminals. The latched signalsare transmitted as CN MSG control network message signals to the flickgenerator 15. The CN MSG signals define forty-eight bits in twelvefour-bit nibbles. In response to the same assertion of the LOAD NEW MSGsignal, the message priority select circuit repeats the above-describedoperations to enable generation of a new word to be loaded into thetransmit message buffer in response to the subsequent assertion of theLOAD NEW MSG signal.

The flick generator 815 successively selects nibbles from the messagebuffer 814, appends to each nibble a tag bit to form five-bit flicks andtransmits them as FLICK OUT (4:0) signals at each of twelve successiveticks of the NODE CLK signal. In addition, the flick generator maintainsa running checksum which it transmits as the thirteenth flick. FIG. 10Cdepicts a detailed block diagram of the flick generator 815.

With reference to FIG. 10C, the flick generator 815 includes a transmittiming control circuit 850 that iteratively generates XMIT 0 transmitflick zero through XMIT 12 transmit flick 12 signals, includingbinary-encoded XMIT 3-10(2:0) transmit flicks three through ten signals.The XMIT 0 through XMIT 12 signals enable other circuitry depicted onFIG. 10C to transmit successive ones of the thirteen flicks comprising acontrol network message packet 60. The transmit timing control circuit850 generates the XMIT 0 through XMIT 12 signals in synchronism withsuccessive ticks of the NODE CLK signal, while enabled by a negated CNSTOP SEND control network stop sending signal and a negated MSG FLOW NOKmessage flow not ok signal, with the proviso that once the transmittiming control circuit 850 has begin transmitting a control networkmessage packet 60 and has sequenced from the XMIT 0 transmit flick zerosignal to the transmit flick one signal, it will continue sequencingthrough the remaining XMIT 2 through XMIT 12 signals, and stop when itrecycles to the XMIT 0 transmit flick zero signal again.

Various ones of the CN MSG control network message signals, representingvarious ones of the nibbles in the transmit message buffer 814, arecoupled to several circuits in the flick generator 815. In particular,the transmit flick generator includes a flick selector circuit 844 that,generally, selects signals from successive nibbles of the transmitmessage buffer for transmission in successive flicks. A flow controlcircuit 845 receives flow control signals from the receiver section 801(described below in further detail in FIG. 10D) that controltransmission of control network message packets by the flick generator815. A data nibble order circuit 846 determines the order in whichnibbles 814(4) through 814(10) will be selected for transmission by theflick selector circuit 844. Finally, a tag signal generator circuit 847generates tag signals to be transmitted as the high-order FLICK OUTsignal.

As noted above, the flow control circuit 845 receives flow controlsignals from the receiver section 801 (described below in further detailin FIG. 10D) that control transmission of control network messagepackets 60 by the flick generator 815 in response to selected tagsignals in control network message packet 60 received by the receiversection 801. The control network node 50(1,j) connected to the controlnetwork interface 204 in each leaf 21 can control the flow of controlnetwork message packets 60 thereto from the same control networkinterface 204 by suitable conditioning of the scan flow bits 72(1)through 72(5), the broadcast user flow bit 73 and the broadcastsupervisor flow bit 74.

For example, if the receiver section 801 receives a control networkmessage packet 60 in which the broadcast supervisor flow bit 74 isclear, the flick generator 815 is disabled from transmitting controlnetwork message packets 60 from information in transmit message buffer814 that originated from the supervisor broadcast transmit FIFO 810.Similarly, if the receiver section 801 receives a control networkmessage packet in which the broadcast user bit 73 is clear, the flickgenerator 815 is disabled from transmitting control network messagepackets 60 from information in transmit message buffer 814 thatoriginated from the broadcast buffer 811. Finally, if the receiversection 801 receives a control network message packet 60 in which a scanflow bit 72(1) through 72(5) is clear, the flick generator 815 isdisabled from transmitting control network message packets 60 frominformation in transmit message buffer 815 that originated from thecombine buffer 812.

In each case, the flick generator 815 will stall the transmit messageselect and assembly circuit 813 and will remain disabled fromtransmitting a message packet 60 containing information originating fromthe particular FIFO 810 through 812 until the receiver section 801receives a control network message packet 60 in which the particular bitis set. However, if the transmit message buffer 814 contains informationthat originated from a different one of FIFOs 810 through 812 than thatfrom which it is disabled, the flick generator 815 may transmit acontrol network message packet 60 using that information.

More specifically, the flow control circuit 845 includes two decoders,namely, a message type decoder 861 and a packet type decoder 862 thatreceive the ones of the CN MSG (3:0) and CN MSG (7:0) control networkmessage signals from nibbles 414(0) and 414(1) of transmit messagebuffer 814. The CN MSG (3:0) signals received by decoder 861 comprisethe ones of the CN MSG signals that contain the message typeinformation. If the CN MSG (3:0) signals have the encoding for amulti-source message, the decoder 861 asserts a MULT SRC MSGmulti-source message signal, and if they have the encoding for a singlesource message the decoder 861 asserts a SINGLE SRC MSG single sourcemessage signal.

The CN MSG (7:4) signals received by decoder 862 comprise the ones ofthe CN MSG signals that contain the packet type information. Inparticular, the PKT TYPE signals identify whether the contents of thetransmit message buffer 814 originated in any of the FIFOs 810 through812, and, if so, which one. The decoder 862 generates three outputsignals, identified as COMB combine, SBC supervisor broadcast and BCbroadcast, which, if asserted, indicate that the contents of messagebuffer originated in the combine transmit FIFO 812, the supervisorbroadcast transmit FIFO 810 or the broadcast transmit FIFO 811,respectively.

The SINGLE SRC MSG single source message and MULT SRC MSG multi-sourcemessage signals from decoder 861, and the COMB, SBC and BC signals fromdecoder 862 are coupled, along with a RCVD BC FLOW received broadcastbuffer flow signal, a RCVD SBC FLOW received supervisor broadcast bufferflow signal and a RCVD SCAN FLOW received scan flow signal to a circuitcomprising a plurality of AND gates 864 through 866 and inverters 870through 875. The RCVD BC FLOW, RCVD SBC FLOW and RCVD SCAN FLOW signalscomprise the flow control signals generated by the receive section 801in response to the respective tag bits of control network messagepackets 60 received from the control network node 50(1,j). If all ofthese signals are asserted, inverters 870 through 872 will disable ANDgates 863 through 865, enabling, in turn, inverters 873 through 875 toenergize AND gate 866 to assert a FLOW PERM flow permitted signal.

On the other hand, if one of the flow control signals from the receivesection 801 is negated, and if the signals from decoders 861 and 862have appropriate conditions, the AND gate 866 will be disabled and theFLOW PERM flow permitted signal will be negated. If, for example, theRCVD SCAN FLOW signal is negated, if the MULT SRC MSG multi-sourcemessage and COMB signals are asserted and inverter 872 will complementthe negated RCVD SCAN FLOW signal to energize the AND gate 865. Theenergized AND gate 865 will enable inverter 875 to disable AND gate 866,causing the FLOW PERM flow permitted signal to be negated. The other ANDgates 863 and 864, and respective inverters connected thereto, operatesimilarly in response to the RCVD BC FLOW and RCVD SBC FLOW signals,respectively if decoder 861 is asserting the SINGLE SRC MSG singlesource message signal.

The FLOW PERM flow permitted signal is coupled to one input terminal ofa multiplexer 867. If the transmit timing control circuit 850 isasserting the XMIT 0 transmit flick zero signal, the multiplexer 867couples the FLOW PERM signal to a flip-flop 868, which latches thesignal at the next tick of the NODE CLK signal. If the FLOW PERM signalis asserted, the flip-flop 868 asserts the MSG FLOW OK signal. The MSGFLOW OK signal is complemented by an inverter 869. Thus, if the FLOWPERM signal is not asserted, causing the flip-flop 868 to negate the MSGFLOW OK signal, inverter 869 asserts a MSG FLOW NOK message flow not oksignal.

The MSG FLOW OK signal from flip-flop 868 is coupled to the other inputof multiplexer 867. It will be appreciated that at subsequent ticks ofthe NODE CLK signal, the XMIT 0 transmit flick zero signal will benegated, enabling the multiplexer 867 to couple the MSG FLOW OK signalback to the data input terminal of the flip-flop 868. Thus, theflip-flop 868 will maintain its condition at subsequent ticks of theNODE CLK signal as the transmit timing control circuit 850 asserts theXMIT i through XMIT 12 signals. At the next assertion of the XMIT 0signal, the multiplexer 867 again couples the FLOW PERM flow permittedsignal to the data input terminal of flip-flop 868, at which point itlatches the condition of the signal at that point.

It will be appreciated that, if the decoder 861 is not asserting theSINGLE SRC MSG single source message signal, the conditions of the RCVDBC FLOW and RCVD SBC FLOW signals will not affect the condition of theFLOW PERM flow permitted signal, and, thus, will not affect thecondition of the MSG FLOW OK and MSG FLOW NOK signals. In addition, ifthe decoder 861 is not asserting the MULT SRC MSG multi-source messagesignal, the condition of the RCVD SCAN FLOW signal will not affect thecondition of the FLOW PERM flow permitted signal. Thus, if, for example,the MSG TYPE signals identify the idle or abstain message packet types,neither the MULT SRC MSG nor the SINGE SRC MSG signal will be asserted,and so the FLOW PERM signal will be asserted, as will the MSG FLOW OKsignal.

The portion of the CN MSG control network message signals from nibble814(2) of the transmit message buffer 814 is coupled to the data nibbleorder circuit 846. That circuit 846 determines the order in which theflick selector circuit 844 transmits signals from nibbles 814(3) through814(10) in the FLICK OUT signals. If the control network message packet60 being transmitted is a multiple source message initiating certaintypes of arithmetic operations, such as addition, the nibbles 70(0)through 70(7) of the packet data portion 62 will carry data ofincreasing significance. This permits the control network nodes 50 toproperly generate carries from one nibble 70(i) to the next nibble70(i+1). On the other hand, in some operations, such as determination ofa maximum, the control network nodes 50 will perform nibble-by-nibblecomparisons of data in nibbles 70(0) through 70(7) of themultiple-source message packets. Accordingly, the successive nibbles70(0) through 70(7) will carry data of decreasing significance.

The binary-encoded XMIT 3-10 (2:0) signals generated by the transmittiming control circuit 850 are coupled to an XOR (exclusive-OR) gatecircuit 880. The XOR gate circuit 880 generates D FLICK SEL (2:0) dataflick select signals representing the exclusive-OR of each of thecorresponding XMIT 3-10 (2:0) signals with a REV DATA reverse datasignal, which is controlled by the data nibble order circuit 846. Thesequence of D FLICK SEL (2:0) data flick select signals determines thesequence of nibbles 814(3) through 814(10) of transmit message buffer814, and thus the order of significance of the data transmitted in thesuccessive nibbles 70(0) through 70(7) in the control network messagepacket 60. If the REV DATA signal is negated, the sequence ofbinary-encoded values of the D FLICK SEL (2:0) signals corresponds tothe sequence of binary-encoded values of the XMIT 3-10 (2:0) signals,and so contents of nibbles 814(3) through 814(10) are transmitted innibbles 70(0) through 70(7), respectively.

On the other hand, if the REV DATA reverse data signal is asserted, theD FLICK SEL (2:0) data flick select signals correspond to the complementof the respective XMIT 3-10 (2:0) signals. In that case, the sequence ofthe binary encoded values of the D FLICK SEL (2:0) signals is thereverse of the sequence of binary-encoded values of the XMIT 3-10 (2:0)signals, and so the nibbles 70(0) through 70(7) of the control networkmessage packet 60 carry the contents of nibbles 814(10) through 814(3).

The data nibble order circuit 846 generates the REV DATA reverse datasignal. Circuit 846 includes a decoder 881 that receives CN MSG (11:8)control network message signals from nibble 814(2) of the transmitmessage buffer 814. If these signals have the encoding to identify amaximum arithmetic operation, the decoder asserts a MAYBE MAX signal,which enables one input of an AND gate 882. If contemporaneously thedecoders 861 and 862 are asserting the MULT SRC MSG multiple sourcemessage signal and COMB combine signal, the control network messagepacket 60 generated from the contents of transmit message buffer 814will enable a maximum arithmetic operation. When that occurs, AND gate882 is energized to assert a MAX signal. When the XMIT 2 transmit flicktwo signal is asserted, a multiplexer 883 couples the MAX signal to thedata input terminal of a flip-flop 884, which latches the MAX signal atthe next tick of the NODE CLK signal.

The flip-flop 884 generates the REV DATA reverse data signal. Inaddition to controlling the XOR gate 880, the REV DATA signal is alsocoupled to the other data input terminal of multiplexer 883, whichcouples that signal to the data input terminal of the flip-flop 884while the XMIT 2 transmit flick two signal is not asserted. It will beappreciated that the condition of the REV DATA reverse data signalsubsequent to the time at which the XMIT 2 transmit flick two signal isasserted will correspond to the condition of the MAX signal at the timethe XMIT 2 signal is asserted.

Accordingly, if any of the MULT SRC MSG multiple source message, COMBcombine and MAYBE MAX signals is negated, which indicates that thecontrol network message packet 60 being transmitted is not enabling amaximum arithmetic operation, the AND gate 882 does not assert the MAXsignal and so the REV DATA signal is not asserted. On the other hand, ifall of the MULT SRC MSG multiple source message, COMB combine and MAYBEMAX signals are asserted, indicating that the control network messagepacket 60 being transmitted is enabling a maximum arithmetic operation,the AND gate 882 will assert the MAX signal and so the REV DATA signalwill be asserted. As noted above, the asserted REV DATA signal enablesthe sequence of binary-encoded values of the D FLICK SEL (2:0) dataflick select signals generated by XOR gate 880 to have the reverse orderthan if the REV DATA signal is negated.

The flick selector circuit 844 receives the CN MSG (47:0) signals atinput terminals of a group of multiplexers 851 through 856 and an ANDgate 855. The output terminals of multiplexers 851 through 854 arecoupled to respective input terminals of a flick select multiplexer 856.The multiplexers 851 through 854 and 856, along with AND gate 855,cooperate to select successive four-bit nibbles for transmission assuccessive flicks of a control network message packet 60.

In particular, CN MSG (3:0) control network message signals identifyingthe message type, transmitted by nibble 814(0) of the transmit messagebuffer 814, are coupled to one set of input terminals of multiplexer851. The other set of input terminals of multiplexer 851 receive IDLEsignals representing the message type encoding for an control networkmessage packet 60 of the idle type. The multiplexer 851 is controlled bythe MSG FLOW NOK message flow not ok signal from the flow controlcircuit 845. If the MSG FLOW NOK signal is asserted, the multiplexer 851couples the IDLE signals to the respective input terminals of the flickselect multiplexer 856. On the other hand, if the MSG FLOW NOK signal isnegated, the multiplexer 851 couples the CN MSG (3:0) signals to thesame input terminals of flick select multiplexer 856.

The CN MSG (7:4) signals, which are transmitted by nibble 814(1) of thetransmit message buffer 814 and identify the packet type, are coupled toinput terminals of AND gate 855. If the message flow circuit 845 isasserting the MSG FLOW OK signal, the AND gate 855 gates CN MSG (7:4)signals to one set of input terminals of the multiplexer 855. The otherset of input terminal receive the CN MSG (11:8) signals, which aretransmitted by nibble 814(2) of the transmit message buffer. If the XMIT2 transmit flick two signal is negated, the multiplexer 852 couples thesignals from AND gate 855 to a set of input terminals of flick selectmultiplexer 856. On the other hand, if the XMIT 2 signal is asserted,the multiplexer 852 couples the CN MSG (11:8) signals thereto.

The multiplexer 853 has eight sets of input terminals, each receivingfour CN MSG signals from one of nibbles 814(3) through 814(10) of thetransmit message buffer. The multiplexer 853 is controlled by the DFLICK SEL (2:0) signals from XOR gate 880. The multiplexer 853 couplessignals at selected sets of inputs, as identified by the D FLICK SEL(2:0) signals, to a set of input terminals of the flick selectmultiplexer 856. The D FLICK SEL (2:0) signals determine the order inwhich the signals from the nibbles 814(3) through 814(10) are coupled tothe flick select multiplexer 856.

Finally, multiplexer 854 has one set of input terminals that receive CNMSG (47:44) signals from the nibble 814(11) of the transmit messagebuffer 814. The CN MSG (47:44) signals represent the contents of theglobals register 820, which, as shown in FIG. 5, are transmitted in thelast flick of a control network message packet 60 before the flickcontaining the checksum. If the XMIT 11 transmit flick eleven signal isasserted, the multiplexer 854 couples the signals from this set ofterminals to a fourth set of input terminals of the flick selectormultiplexer 856. When the XMIT 11 signal is not asserted, themultiplexer 854 couples CHECK (3:0) signals from a checksum generator857 to the same set of input terminals.

The flick select multiplexer 856 is controlled by the XMIT 0 throughXMIT 12 signals from transmit timing control circuit 850. When the XMIT0 signal is asserted, the flick select multiplexer 856 couples thesignals from multiplexer 851 as the low-order FLICK OUT (3:0) signals.If the MSG FLOW NOK signal is negated, the flick includes the contentsof nibble 814(0) of the transmit message buffer 814. However, if the MSGFLOW NOK signal is asserted, the flick includes the idle message packettype code.

The sequential assertion of the XMIT 1 and XMIT 2 signals enables theflick select multiplexer 856 to couple the signals from multiplexer 852as the low-order FLICK OUT (3:0) signals for the next two flicks. If theMSG FLOW OK signal is asserted, the flicks include the contents ofnibbles 814(1) and 814(2) of the transmit message buffer 814. If,however, the MSG FLOW OK signal is negated, the signals from multiplexer851 while the XMIT 1 signal is asserted will all be negated, and so theFLICK OUT (3:0) signals at the point will also be negated.

The assertion of the XMIT 3-10(2:0) signals and the XMIT DATA signalenables the flick select multiplexer 856 to couple signals frommultiplexer 853 as the low-order FLICK OUT (3:0) signals for the nexteight flicks. The flicks will include the contents of nibbles 814(3)through 814(10) of the transmit nibble buffer 814, with the orderdepending on the condition of the REV DATA signal.

Finally, the sequential assertion of the XMIT 11 and XMIT 12 signalsenables the flick select multiplexer to couple the signals frommultiplexer 854 as the low-order FLICK OUT (3:0) signals for the finaltwo flicks. When the XMIT 11 signal is asserted, the flick includes thecontents of the nibble 814(11) of the transmit message buffer 814. Onthe other hand if the XMIT 12 signal is asserted, the flick includes thelow-order CHECK (3:0) checksum signals from checksum generator 857.

The tag signal generator circuit 847 receives signals from a number ofsources and, in response to each the XMIT (0) through XMIT (12) signals,selects one to couple as a SEL TAG selected tag signal. The SEL TAGsignal is transmitted to the control network node 50(1,j) and to thechecksum generator 857 as the FLICK OUT (4) signal. In particular, thetag signal generator circuit 847 couples a COM RCV FIFO NR FULL combinereceive first-in first-out buffer near full signal, from buffer 835(FIG. 10A-1) as the SEL TAG signal in response to the XMIT 0 signal andencodings of the XMIT 3-10(2:0) signals having the values three, six,eight and ten. This signal provides the scan flow bits 72(1) through72(5) of the control network message packet 60.

The tag signal generator circuit 847 couples as the SEL TAG signal asignal representing the segment flag 825, the BC RCV FIFO NR FULLbroadcast receive first-in first-out buffer nearly full signal, and theSBC RCV FIFO NR FULL supervisor broadcast receive first-in first-outbuffer nearly full signal in response, respectively, to the XMIT 1 andXMIT 2 signals and the encoding of the XMIT 3-10 (2:0) signals havingthe value one. These signals provide the segment, broadcast user flow,and broadcast supervisor user flow bits 77, 73 and 74 in the controlnetwork message packet 60 (FIG. 5).

In addition, the circuit 847 couples as the SEL TAG signal a signalrepresenting the condition of the all-fall-down mode flag 824 and flushregister 821 in response to the encodings of the XMIT 3-10 (2:0) signalsrepresenting the values two and four. These signals provide theall-fail-down mode bit 81 and flush bit 75 in the control networkmessage packet 60 (FIG. 5).

Finally, tag signal generator circuit 847 couples as the SEL TAGselected tag signal the CHECK (4) high-order checksum signal from thechecksum generator 857. This provides the high-order bit of the checksumflick 63 in the control network message packet 60 (FIG. 5).

As noted above, the checksum generator 857 generates CHECK (4:0)checksum signals representing a checksum value. The checksum generator857 generates the checksum over the first twelve flicks of the controlnetwork message packet 60. The checksum generator 857 is reset by theassertion of the XMIT 0 transmit flick zero signal and updates thechecksum value in response to each successive tick of the NODE CLKsignal.

The transmit section 800 couples the FLICK OUT (4:0) flick outputsignals to the control network node 50(1,j) connected thereto, and alsoto the root control/loopback section 802. As noted above, if the rootcontrol/loopback portion is asserting the ROOT UP signal, which occursif the control network interface 204 is a logical root, it buffers theflicks defining control network message packets 60 for transmission tothe receive section 801.

c. Receive Section 801

The receive section 801 includes several elements, including the flickdemultiplexer 830 which receives the sequential flicks of a controlnetwork message packet 60 and assembles them in the receive messagebuffer 831. The destination control 832 determines one of severaldestinations for the contents of the receive message buffer, includingthe FIFOs 833 through 835 and interrupt register 822, and coupled thecontents to the identified destination. Details of the flickdemultiplexer 830 and receive message buffer 831 will be described inconnection with FIG. 10D, and the destination control will be describedin connection with FIG. 10E.

With reference to FIG. 10D, the flick demultiplexer 830 includes a flicksource multiplexer 890 that selects from among two sources of signalsfor control network message packets 60 in response to a SEL XMIT selecttransmit section signal from the root control/loopback section 802. Inparticular, if the SEL XMIT signal is negated, the flick sourcemultiplexer 890 couples the FLICK IN (4:0) flick input signals, which itreceives from the control network node 50(1,j) connected thereto at oneset of input terminals, as RCVD FLICK (4:0) received flick signals. Onthe other hand, if the SEL XMIT signal is asserted, which generallyoccurs when the control network interface 204 is a logical root, themultiplexer 890 couples the BUF FLICK OUT (4:0) buffered flick outputsignals from the flick buffer 816 as the RCVD FLICK (4:0) received flicksignals.

In either case, the RCVD FLICK (4:0) received flick signals are coupledto several portions of the flick demultiplexer 830. In particular, theRCVD FLICK (4:0) signals are connected to a receive timing generatorcircuit 891, a check circuit 892, and a flick distribution and latchcircuit 893. The receive timing generator circuit 891, like the transmittiming control circuit 850, iteratively generates RCV 0 receive flickzero through RCV 12 receive flick 12 signals. The RCV 0 through RCV 12signals enable other circuitry depicted on FIG. 10D to receive low-orderRCVD FLICK (3:0) signals corresponding successive ones of the firsttwelve flicks of a control network message packet 60, and to latch theflicks in respective nibbles 831(0) through 831(11) of the receivemessage buffer 831. In addition, the RCV 0 through RCV 12 signals enableother circuitry to latch successive high-order RCVD FLICK (4) signalsrepresenting the tag bit in each flick and to direct the latched signalsto appropriate destinations.

The receive timing generator circuit 891 includes a receive timingcontrol circuit 894 that actually generates the RCV 0 through RCV 12timing signals. If all of the RCVD FLICK (4:0) received flick signalsare negated, a NOR gate 895 enables one input of an AND gate 896. If thereceive timing control circuit 894 is asserting the RCV 0 timing signal,and if all of the RCVD FLICK (4:0) signals are negated, AND gate 896 isenergized, which, in turn, energizes an OR gate 897 to assert a RST TOST 0 reset to state zero signal. The RST TO ST 0 signal is coupled to areset terminal of the receive timing control circuit 894 to enable thecircuit 894 to continue asserting the RCV 0 signal.

The first flick of a control network message packet is identified by atleast one of the RCVD FLICK (4:0) signals being asserted when thereceive timing control circuit 894 is asserting the RCV 0 receive zerosignal. When that occurs, the NOR gate 895 disables the AND gate 895, inturn disabling the OR gate 897 and negating the RST TO ST 0 reset tostate zero signal. This enables the timing control circuit to beginstepping through the RCV 0 through RCV 12 signals in synchronism withsuccessive ticks of the NODE CLK signal. It will be appreciated that theflick demultiplexer 830 will receive RCVD FLICK (4:0) signalsrepresenting all thirteen flicks of the control network message packet60 as the receive timing control circuit 894 is stepping through the RCV0 through RCV 12 signals.

When the receive timing control circuit 894 asserts the RCV 12 signal,that signal energizes the OR gate 897 to again assert the RST TO ST 0reset to state zero signal, which enables the circuit 894 to reset andassert the RCV 0 signal at the next tick of the NODE CLK signal. At thispoint, if the RCVD FLICK (4:0) signals are all negated, indicating thatthe first flick of a new control network message packet 60 is not thenbeing received, the NOR gate 895 is energized and it and the assertedRCV 0 signal maintain the OR gate 897 in the energized condition. Thereceive timing control circuit 894 thus maintains the RCV 0 signal inthe asserted condition until at least one of the RCVD FLICK (4:0)signals is asserted.

As with transmissions of data nibbles 70(0) through 70(7) in successiveflicks by the flick generator 815 as described above, the order ofnumerical significance of the received nibbles 70(0) through 70(7) of acontrol network message packet received by the flick demultiplexer 830will depend upon whether the control network message packet 60 is amultiple source message with the results of a maximum arithmeticoperation. If not, the RCV REV DATA signal is negated, which enables aseries of multiplexers 900(3) through 900(10) to assert SEL RCV 3through SEL RCV 10 timing signals in the same sequence as the RCV 3through RCV 10 timing signals. On the other hand, if the control networkmessage packet 60 being received is a multiple source message with theresults of a maximum arithmetic operation, the RCV REV DATA signal isasserted, which enables the multiplexers 900(3) through 900(10) toassert the SEL RCV 3 through SEL RCV 10 signals in the reverse order asthe RCV 3 through RCV 10 receive timing signals.

The check circuit 892 includes two elements. A checksum check circuit902 iteratively receives the RCVD FLICK (4:0) signals representingsuccessive flicks of a control network message packet 60. The checksumcheck circuit 902 is reset in response to the RCV 0 signal from receivetiming control circuit 894, and receives the successive flicks at eachtick of the NODE CLK signal thereafter. In response to the RCV 12signal, which is asserted at the point at which the RCVD FLICK (4:0)signals represent the checksum portion 63 of the received message packet60, if the checksum check circuit 902 determines that the message packetwas properly received, it asserts the CHECK OK signal.

A protocol check circuit 903 iteratively receives the low-order RCVDFLICK (3:0) signals in response to the RCV 0 through RCV 2 signals, thatis, the signals representing the flicks comprising packet header 61 of acontrol network message packet 60, and determines whether the encodingscorrespond to those that are permissible in the particular system 10. Ifthe protocol check circuit 903 determines that the encodings arepermissible, it asserts a PROT OK protocol ok signal. In addition, theprotocol check circuit 903 determines whether the message packet 60being received is of the multiple source type and has the result of amaximum arithmetic operation, and if so, it asserts the RCV REV DATAreceive reverse data signal, which is coupled to XOR gate 900.

The low-order RCVD FLICK, (3:0) received flick signals are also coupledto data input terminals of a series of multiplexers 904(0) through904(11) [generally identified by reference numeral 904(i)]. Each of themultiplexers 904(0) through 904(2) and 904(11) is controlled by one ofthe respective RCV 0 through RCV 2 and RCV 11 receive timing signalsfrom receive timing generator circuit 891. When the respective RCV 0through RCV 2 and RCV 11 receive timing signals are asserted, therespective multiplexer 904(i) couples the RCVD FLICK (3:0) signals toinput terminals of a respective latch generally identified by referencenumeral 905(i), which latch the signals at the next tick of the NODE CLKsignal. If the respective RCV "i" receive timing signal is negated, themultiplexers 904(i) couple the latched signals from the respectivelatches 905(i) back to their input terminals, to enable the latches willmaintain their conditions at subsequent ticks of the NODE CLK signal.

The multiplexers 904(3) through 904(10), on the other hand, arecontrolled by the SEL RCV 3 through SEL RCV 10 selected receive timingsignals from multiplexers 900(3) through 900(10). When the SEL RCV 3through SEL RCV 10 signals are asserted, multiplexers 904(3) through904(10) couple the RCVD FLICK (3:0) signals to input terminals of arespective latch 905(3) through 905(10), which latch the signals at thenext tick of the NODE CLK signal. Accordingly, if the protocol checkcircuit 903 is negating the RCV REV DATA receive reverse data signal,the RCVD FLICK (3:0) signals received during sequential assertions ofthe RCV 3 through RCV 10 signals will be latched into the successivelatches 905(3) through 905(10). On the other hand, if the protocol checkcircuit 903 is asserting the RCV REV DATA receive reverse data signal,the RCVD FLICK (3:0) signals received during sequential assertions ofthe RCV 3 through RCV 10 signals will be latched into the successivelatches in the reverse order, that is, into latches 905(10) through905(3).

The CHECK OK and PROT OK signals from check circuit 892, along with theRCV 12 signal are coupled to a receive strobe enable circuit 907. If,during receipt of the RCVD FLICK (4:0) signals representing successiveflicks of the control network message packet 60, check circuit 892asserting both the CHECK OK and the PROT OK signals, the RCV 12 signalenables the receive strobe enable circuit 907 to assert a LOAD RCVD MSGload received message signal, which enables the receive message bufferto latch the signals coupled to its input terminals by the latches905(i). The received message buffer 831 transmits the latched signals asRCVD MSG (47:0) received message signals to the destination controlcircuit 832.

The flick demultiplexer 815 also includes a number of tag signal latchcircuits 910(i), some of which are also depicted on FIG. 10D. The index"i" in reference numeral 910(i) corresponds to the index "i" of thereceive timing signal RCV "i" that is asserted when the particular tagsignal is being received. The tag signal latch circuits are all similar,and so only circuit 910(2) that generates the RCVD BC FLOW receivedbroadcast flow signal, circuit 910(4) RCVD SBC FLOW received supervisorbroadcast flow signal and circuit 910(0,3,6,8,10) that generates theRCVD SCAN FLOW received scan flow signal, which are used to control thetransmit flick generator 815 (FIG. 10C), are depicted in FIG. 10D.

As shown in FIG. 10D, each tag signal latch circuit 910(i) includes amultiplexer 911(i), which receives at one input terminal the RCVDFLICK(4) received flick signal representing the tag bit of each flick.Each multiplexer 911(i) is controlled by the corresponding RCV "i"receive timing signal to couple the RCVD FLICK (4) signal to the datainput terminal of a flip-flop 912(i). The tag signal latch circuit910(0,3,6,8,10) also includes an OR gate 913 that asserts a RCV(0,3,6,8,10) signal when any of the RCV 0, RCV 3, RCV 6, RCV 8, or RCV10 signals are asserted to then enable multiplexer 911(0,3,6,8,10) tocouple the RCVD FLICK (4) signal to the data input terminal of flip-flop912(0,3,6,8,10). With reference to FIG. 5, it will be appreciated that,when the RCV 0, RCV 3, RCV 6, RCV 8, and RCV 10 signals are asserted,the RCVD FLICK (4) signal representing the flicks received at thosepoints in time will represent the scan flow bits 72(1) through 72(5),which is latched in tag signal latch circuit 910(0,3,6,8,10).

Each flip-flop 912(i) is clocked in response to the NODE CLK signal.Thus, the tag signal as latched by each tag signal latch circuit 910(i)is updated at the next tick of the NODE CLK signal after the assertionof the RCV "i" signal. The output signal from each flip-flop 912(i) iscoupled to the other data input terminal of the associated multiplexer911(i), which couples that signal back to the data input terminal of theflip-flop 912(i) when the RCV "i" receive timing signal is not asserted.Accordingly, each tag signal latch circuit 910(i) is updated at the nexttick of the NODE CLK signal following the associated RCV "i" receivetiming signal, and maintains the state until the next update.

The destination control circuit 832, when an control network messagepacket. 60 is received, identifies the particular FIFO 833, 834 or 835,or the one of registers 804 into which the contents of the receivemessage buffer 831 are to be stored, and enables the contents to bestored therein. With reference to FIG. 10E, the destination controlcircuit 832, includes a destination decode circuit 914 which receivesthe RCVD MSG (3:0) and RCVD MSG (7:4) received message signals from thereceive message buffer 831. These signals represent the contents of,respectively, the message type field 64 and packet type field 65 of thereceived control network message packet 60.

In response to the assertion of the LOAD RCVD MSG load received messagesignal from the receive strobe enable circuit 907, the destinationdecode circuit 914 uses these RCVD MSG signals to select for assertionone of a PUSH RCV BC FIFO push receive broadcast first-in first-outbuffer signal, a PUSH RCV SBC FIFO push receive supervisor broadcastfirst-in first-out buffer signal, a PUSH RCV COM FIFO push receivecombine first-in first-out buffer signal and a STORE INTR REG storeinterrupt register signal. The asserted signal enables the contents ofnibbles 831(3) through 831(10) of received message buffer 831, whichcorrespond to the contents of data nibbles 70(0) through 70(7) of thereceived message packet 60 and which are represented by the RCVD MSG(43:12) received message signals, to be stored in one of the FIFO 833through 835 or in interrupt register 822.

In addition, the destination decode circuit 814 also generates a STOREGLOBALS signal that enables the globals register 820 to latch the RCVDMSG (47:44) received message signals. These signals represent thecontents of nibble 831(11) of received message buffer 831, which, inturn, correspond to the contents of the global information nibble 71 ofthe receive control network message packet 60.

d. Root Control/Loopback Section 802

FIGS. 10F and 10G depict details of the root control/loopback section802. In particular, FIG. 10F depicts a detailed logic diagram of theroot flag control circuit 817. With reference to FIG. 10F, the root flagcontrol circuit 817 includes three general portions, including a decodercircuit 920, a root establishment timing circuit 921 and a root deletiontiming circuit 922. The decoder circuit 920 determines whether thecontrol network interface is transmitting a control network messagepacket 60 of the single source message type and of the configurationpacket type, and, if so, determines whether the height value containedtherein identifies the control network interface 204 or a controlnetwork node 50(i,j) as a logical root.

If the decoder circuit 920 determines whether the control networkinterface 204 is transmitting a control network message packet 60 toestablish the control network interface 204 as a logical root, and ifthe control network interface 204 is not then a logical root, the rootestablishment timing circuit 921 establishes the control networkinterface 204 as a logical root, and asserts the ROOT UP signal.Thereafter, the receive section 801 is enabled to receive subsequentcontrol network message packets 60 from the transmitter section 800 ofthe control network interface 204. On the other hand, if the decodercircuit 920 determines that the control network interface istransmitting a control network message packet 60 to establish a controlnetwork node 50(i,j) as a logical root, and if the root establishmenttiming circuit 921 is currently asserting the ROOT UP signal, the rootdeletion timing circuit 922 enables the root establishment timingcircuit 921 to negate the ROOT UP signal, so that the receive sectionwill receive subsequent control network message packets 60 from thecontrol network node 50(1,j) connected thereto. The root establishmenttiming circuit 921 and the root deletion timing circuit 922 operate soas to ensure that the ROOT UP signal is asserted and negated at controlnetwork message packet boundaries, that is, after the receive sectionhas finished receiving a control network message packet 60 that it isthen receiving. This ensures that the receive section 801 does notmisinterpret the FLICK IN (4:0) signals representing successive flicksof a control network message packet 60 that it is receiving.

If the CN MSG (3:0)-signals indicate that the control network messagepacket is of the single source message type and the CN MSG (7:4) signalsindicate that the packet is of the configuration type, and if the CN MSG(16:12) signals have a binary-encoded value of zero, the control networkinterface 204 is to become a logical root. The decoder circuit 920includes an inverter 923 that receives the CN MSG (16:12) signals,representing the contents of the first nibble 70(0) (FIG. 5) and one bitof the second nibble 70(1) of the packet data portion 62 of the messagepacket 60 to be transmitted. The inverter 924 couples complements ofthese signals to input terminals of an AND gate 924. If all of the CNMSG (16:12) signals are negated, which will occur if they have thebinary-encoded value of zero, the AND gate 924 asserts a ROOT HT 0 rootheight zero signal.

The decoder circuit 920 also includes an OR gate 928 that also receivesthe CN MSG (16:12) control network message signals and asserts a ROOT HTNE 0 root height not equal to zero signal if at least one of the CN MSG(16:12) signals is asserted. It will be appreciated that, if at leastone of the CN MSG (16:12) signals is asserted, the binary-encoded valuethereof is not zero. Accordingly, if the CN MSG (3:0) and CN MSG (7:4)signals indicate that the control network message packet 60 is of thesingle source message type and the configuration packet type, thecontrol network message packet 60 is establishing a control network node50(i,j) as a logical root. If that occurs while the control networkinterface 204 is a logical root, the root flag control circuit 817 willdelete, that is, disestablish, the root condition at the control networkinterface 204.

It will be appreciated that the CN MSG (16:12) signals may all benegated in connection with control network message packets 60 havingmessage types other than single source or packet types other thanconfiguration. Accordingly, the decoder circuit 920 includes a decoder925 that receives the CN MSG (3:0) control network message signals andthe CN MSG (7:4) signals, which identify the message type and the packettype, respectively. If the ROOT HT 0 signal is asserted, and the CN MSG(3:0) and CN MSG (7:4) signals identify the single source message typeand the configuration packet type, the decoder 925 asserts a CNI ROOT HTcontrol network interface root height signal in response to theassertion of the XMIT 0 transmit flick zero timing signal.

The CNI ROOT HT signal is coupled to the root establishment timingcircuit 921, specifically to an input terminal of an AND gate 927. Thesecond input terminal of AND gate 927 is controlled by a LEAF ST leafstate signal from the root deletion timing circuit 922. If the LEAF STsignal is asserted, the control network interface 204 is not then alogical root. In that case, the asserted CNI ROOT HT signal energizesAND gate 927, which, in turn, energizes an OR gate 930, enabling it toassert a CNI BECOME ROOT control network interface become root signal.The CNI BECOME ROOT signal is coupled to the data input terminal of aflip-flop 931, which is set in response to the next tick of the NODE CLKsignal to assert a CNI BECOME ROOT ST control network interface becomeroot state signal.

The CNI BECOME ROOT ST control network interface become root statesignal is coupled to one input terminal of an AND gate 932. While theRCV 12 receive flick twelve signal is not asserted, the other inputterminal of AND gate 932 is enabled, and the assertion of the CNI BECOMEROOT ST signal causes the AND gate 932 to be energized. While the ANDgate 932 remains energized, the OR gate 930 also remains energized to,in turn, maintain the CNI BECOME ROOT signal in its asserted condition.

When the RCV 12 receive timing signal is next asserted, it and theasserted CNI BECOME ROOT ST signal energize an AND gate 933, which, inturn, energizes an OR gate 934 to assert a WAIT FOR XMIT wait fortransmit state signal. It will be appreciated that this assertion of theRCV 12 receive timing signal occurs when the receive section 801 isreceiving the FLICK IN signals representing the last flick of thecontrol network message packet it is then receiving. The WAIT FOR XMITsignal is coupled to the data input terminal of a flip-flop 935, whichis set at the next tick of the NODE CLK signal to assert a WAIT FOR XMITST wait for transmit state signal. The assertion of the RCV 12 signalalso disables AND gate 932, which, in turn, disables OR gate 930 tonegate the CNI BECOME ROOT signal. The negated CNI BECOME ROOT signalresets the flip-flop 931 at the same tick of the NODE CLK signal, which,in turn, negates the CNI BECOME ROOT ST signal.

With reference to FIG. 10G, which depicts circuitry in the flick buffer816, the asserted WAIT FOR XMIT ST wait for transmit state signalgenerated by flip-flop 935 energizes an OR gate 936 to assert the SELXMIT select transmit signal. As described above in connection with FIG.10D, the assertion of the SEL XMIT select transmit signal enables themultiplexer 890 to receive and use the BUF FLICK OUT buffered flick outsignals from the flick buffer 816, instead of the FLICK IN signals fromthe control network node 50(1,j) connected thereto. At this point, theBUF FLICK OUT signals to the receive section 801 are all negated; and sothe flick buffer 816 is essentially providing the equivalent of controlnetwork message packets of the nil packet type, as described above.However, it will be appreciated that, since this occurs in response tothe RCV 12 signal, this operation occurs only after the receive sectionhas finished receiving a complete control network message packet 60 fromthe control network node 50(1,j) connected thereto.

Returning to FIG. 10F, the WAIT FOR XMIT ST wait for transmit statesignal from flip-flop 935 is coupled to two AND gates 937 and 940. Ifthe WAIT FOR XMIT ST signal is asserted, and if the XMIT 12 transmittiming signal is negated, which is the case before the transmit section800 has finished transmitting a control network message packet 60 it iscurrently transmitting, the AND gate 937 is energized, which in turnenergizes the OR gate 934 to maintain the WAIT FOR XMIT wait fortransmit signal asserted. It will be appreciated that, at this point,the control network message packet 60 being transmitted by the transmitsection 800 is the configuration packet that identifies the controlnetwork interface 204 as the logical root.

When the transmit section 800 is transmitting the last flick of thecontrol network message packet 60, it asserts the XMIT 12 signal. Thecoincidence of the asserted XMIT 12 signal and the asserted WAIT FORXMIT ST wait for transmit state signal energize an AND gate 940, which,in turn, energizes an OR gate 941 to assert a ROOT signal. The ROOTsignal is coupled to the data input terminal of a flip-flop 942. At thenext tick of the NODE CLK signal, flip-flop 942 is set, to assert a ROOTST root state signal. A driver 943, connected to the data outputterminal of the flip-flop 942 receives the ROOT ST signal and couples itas the ROOT UP signal to the control network node 50(1,j) connected tothe control network interface 204. The asserted XMIT 12 signal alsodisables AND gate 937, which, in turn, disables OR gate 934, causing itto negate the WAIT FOR XMIT signal. The negation of the WAIT FOR XMITsignal, in turn, causes the flip-flop 935 to be reset in response to thesame tick of the NODE CLK signal, negating the WAIT FOR XMIT ST signal.

With reference again to FIG. 10G, the asserted ROOT ST signal maintainsthe OR gate 936 in the energized condition, which, in turn, maintainsthe SEL XMIT select transmit signal in the asserted condition.Accordingly, the multiplexer 890 in the flick demultiplexer 830 of thereceive section 801 continues to couple the. BUF FLICK OUT bufferedflick out signals from the flick buffer 816 as the RCVD FLICK (4:0)received flick signals. The asserted ROOT ST signal also energizes an ORgate 944 to assert a GATE FLICK signal, which enables a gate 945 tocouple the FLICK OUT (4:0) signals from the transmit section 800 asGATED FLICK (4:0) signals to the data input terminals of a buffer 946.The buffer 946 latches the GATED FLICK (4:0) signals in response to eachtick of the NODE CLK signal, and provides at its data output terminalsthe BUF FLICK OUT (4:0) buffered flick out signals that are coupled tothe receive section 801.

It will be appreciated that this occurs as the transmitter section 800is beginning transmission of a new control network message packet 60.Accordingly, the receive section 801 will begin receiving a controlnetwork message packet 60, other than the packets of the nil packettype, from the transmit section 800 at the first flick of the respectivepacket. As noted above, the receive section 801 can identify the firstflick as the first flick for which the RCVD FLICK signals are not allnegated.

Returning to FIG. 10F, as noted above, if the transmit message buffer814 contains a control network message packet 60 of the single sourcemessage type and configuration packet type which identifies a heightother than zero as the root level, the decoder 925 asserts the CNI NEROOT HT control network interface not equal to root height signal, andotherwise negates the signal. If the CNI NE ROOT HT signal is negated,an inverter 950 enables one input of an AND gate 947. If the flip-flop942 is asserting the ROOT ST signal, the AND gate 947 is energized,which maintains OR gate 941 in an energized condition and the ROOTsignal asserted. The asserted ROOT signal, in turn, maintains theflip-flop 942 set and the ROOT ST and ROOT UP signals asserted. Whilethe ROOT ST signal is asserted, the energized OR gate 936 (FIG. 10G)maintains the SEL XMIT signal asserted and the gate 945 enabled so thatthe receive section 801 receives the FLICK OUT signals from the transmitsection 800 representing the successive flicks of control networkmessage packet.

When the decoder 925 asserts the CNI NE ROOT HT signal, the coincidenceof that signal and the asserted ROOT ST signal energize an AND gate 951,which, in turn, energizes an OR gate 952 to assert a LEAVE ROOT signal,which is coupled to the data input terminal of a flip-flop 953. Theasserted LEAVE ROOT signal causes the flip-flop 953 to be set at thenext tick of the NODE CLK signal, enabling the flip-flop 953 to assert aLEAVE ROOT ST leave root state signal. Contemporaneously, the assertedCNI NE ROOT HT signal causes inverter 950 to disable ANI) gate 947,which, in turn, disables the OR gate 941, causing it to negate the ROOTsignal. The negated ROOT signal enables the flip-flop 942 to reset atthe same tick of the NODE CLK signal.

The asserted LEAVE ROOT ST leave root state signal also maintains the ORgate 936 (FIG. 10G) energized to assert the SEL XMIT select transmitsignal and OR gate 944 energized to enable gate 945. Accordingly, whilethe LEAVE ROOT ST signal is asserted, the receive section 801 continuesto receive and use BUF FLICK OUT (4:0) buffered flick out signalsrepresenting successive flicks of a control network message packet.

The LEAVE ROOT ST leave root state signal will remain asserted until thereceive section 801 asserts the RCV 12 signal, indicating it hasreceived BUF FLICK OUT (4:0) buffered flick out signals representing thelast flick of a control network message packet 60. In particular, priorto assertion of the RCV 12 signal, the coincidence of the negated RCV 12signal and the asserted LEAVE ROOT ST leave root state signal maintainan AND gate 954 in an energized condition, which, in turn, maintains ORgate 952 energized to assert the LEAVE ROOT signal. As above, the whilethe LEAVE ROOT signal remains asserted at successive ticks of the NODECLK signal, the flip-flop 953 remains set to maintain the LEAVE ROOT STsignal asserted.

However, when the receive section 801 asserts the RCV 12 signal, the ANDgate 954 is disabled, which, in turn, disables the OR gate 952, causingit to negate the LEAVE ROOT signal. The negated LEAVE ROOT signal causesthe FLIP-FLOP 953 to be reset, which, in turn, negates the LEAVE ROOT STsignal. The negated LEAVE ROOT ST signal, in turn, disables OR gate 936,enabling it to negate the SEL XMIT select transmit signal. The negatedSEL XMIT signal enables the multiplexer 890 of flick demultiplexer 830to begin coupling the FLICK IN (4:0) signals from the control networknode 50(1,j) connected thereto as the RCVD FLICK (4:0) received flicksignals. Prior to this point, while the control network interface 204had been asserting the ROOT UP signal, the control network node 50(1,j)had been transmitting FLICK IN (4:0) signals all of which were negated,essentially transmitting control network message packets 60 of the nilpacket type. After negation of the ROOT UP signal, the control networknode 50(1,j) begins transmitting control network message packets 60 ofother message types. As described above, the flick demultiplexer 830identifies the first flick of one of these control network messagepacket 60 as that is the first flick for which the RCVD FLICK (4:0)signals are not all negated.

In addition, the negation of the LEAVE ROOT ST leave root state signalalso disables OR gate 944, causing it to negate the GATE FLICK signal todisable gate 945. Thereafter, until the GATE FLICK signal is againasserted in response to assertion of the ROOT ST root state signal, theGATED FLICK (4:0) signals latched by the buffer 946 will all be negated.

Returning to FIG. 10F, the assertion of the RCV 12 receive timing signalwhile the flip-flop 953 is asserting the LEAVE ROOT ST signal alsoenergizes an AND gate 955. The energized AND gate 955, in turn,energizes an OR gate 956 which, in turn, asserts a LEAF signal, which iscoupled to the data input terminal of a flip-flop 957. The flip-flop 957is set at the next tick of the NODE CLK signal, enabling it to assert aLEAF ST signal. As noted above, the LEAF ST signal controls one inputterminal of AND gate 927.

The asserted LEAF ST signal also enables one input terminal of an ANDgate 961, whose other input terminal is controlled by the CNI EQ ROOT HTcontrol network interface equals root height signal through an inverter960. While the LEAF ST signal is asserted and the CNI EQ ROOT HT signalis negated, the AND gate 961 remains energized, in turn enabling the ORgate to remain energized to maintain the LEAF signal asserted. While theLEAF signal is asserted, successive ticks of NODE CLK signal maintainthe flip-flop 957 in the set condition, maintaining, in turn, the LEAFST signal asserted. With the LEAF ST signal in the asserted condition,when the CNI EQ ROOT HT signal is asserted, the AND gate 927 can beenergized to initiate the sequence described above.

Assertion of the CNI EQ ROOT HT signal also, through inverter 960,disables AND gate 961, which, in turn, disables OR gate 956 causing theLEAF signal to be negated. The negation of the LEAF signal causes theflip-flop 957 to be reset, which, in turn, causes the LEAF ST signal tobe negated at the next tick of the NODE CLK signal. Accordingly, the ANDgate 927 is disabled, by negation of the LEAF ST signal, on tick of theNODE CLK signal after the CNI EQ ROOT HT signal is asserted, and is notenabled again until the LEAF ST signal is again asserted. It will beappreciated that the AND gate 927 is provided so that, if the controlnetwork interface 204 is already a logical root, which will be the caseif the LEAF ST signal is negated, the rest of the root establishmenttiming circuit 921 will be inhibited from sequencing if the CNI EQ ROOTHT signal again is asserted without intervening assertion of the CNI NEROOT HT signal, as the sequencing may cause a momentary glitch or noisein the ROOT UP signal.

C. Data Router Node 22 1. General

FIG. 11A is a general block diagram of a data router node 22 used in thedata router described above, and FIGS. 11B-1 through 11D comprisedetailed block and logic diagrams of the data router node 22. Withreference to FIG. 11A, the data router node 22 includes a childinterface 1001, a parent interface 1002 and a switch 1003, allcontrolled by a node control circuit 1004. The data router node alsoincludes a diagnostic network interface 1005, which provides aninterface to the diagnostic network 16. In addition, the data routernode 22 includes a clock buffer 1008 that receives the SYS CLK systemclock signal from the clock circuit 17 and generates a NODE CLK nodeclock signal in response. In one particular embodiment, the clock buffer1008 comprises a buffer as described in aforementioned Hillis, et al.,patent appn. Ser. No. 07/489,079, filed Mar. 5, 1990, entitled DigitalClock Buffer Circuit Providing Controllable Delay.

The child interface 1001 includes a set of child interface modulesgenerally identified by reference numeral 1001(i) ("i" being aninteger). Each child interface module 1001(i) is connected to a child inthe data router 15, which may comprise either a leaf 21, in the case ofa data router node 22(1,j,k) in the first level of the data router 15,or a data router node 22(i-1,j,k) that forms part of a child data routernode group 20(i-1,j) in the case of data router nodes at higher levels.Each child interface module 1001(i) receives data router message packets30 from the child connected thereto and couples them to the switch 1003.In addition, each child interface module 1001(i) receives data routermessage packets from the switch 1003 and couples them to the childconnected thereto. It will be appreciated that the number of childinterface modules 1001(i) in a child interface 1001 of a particular datarouter node 22(i,j,k) will generally depend upon the fan-out from onelevel "i" to the next lower level "i-1" in the data router 15. In thesystem 10 described herein, the fan-out is four, and the child interface1001 depicted in FIG. 11A includes four child interface modules 1001(0)through 1001(3).

The parent interface 1002 also includes a set of parent interfacemodules generally identified by reference numeral 1002(i) ("i" being aninteger). In data router nodes 22(i,j,k) at levels below the level ofthe root data router node group 20(M,0), each parent interface module1002(i) is connected to a data router node 22(i+1,j,k) that forms partof a parent data router node group 20(i+1,j). In the case of data routernodes 22(M,0,k) in the root data router node group 20(M,0), those nodesdo not require any parent interface modules 1002(i). Each parentinterface module 1002(i) receives data router message packets 30 fromthe parent, if any, connected thereto and couples them to the switch1003. In addition, each parent interface module 1002(i) receives datarouter message packets from the switch 1003 and couples them to theparent, if any, connected thereto. The number of parent interfacemodules 1002(i) in a parent interface 1002 of a particular data routernode 22(i,j,k) will generally depend upon the fan out from one level "i"to the next higher level "i+1" in the fat-tree defining the data router15. In the system 10 described herein, the fan-out at some levels is twoand at other levels is four. The parent interface depicted in FIG. 11Aincludes four parent interface modules 1002(0) through 1002(3), whichwill accommodate a fan-out of four.

It will be appreciated that in other embodiments of system 10, thefan-out, both up and down the fat-tree, may be different, in which casedifferent numbers of child interface modules 1001(i) and parentinterface modules 1002(i) may be provided in a particular data routernode 22(i,j,k). In addition, if, as is the case with one embodiment, allof the circuitry for the data router node 22(i,j,k) is fabricated on asingle integrated circuit chip, the circuit may include a number ofchild interface modules 1001(i) and parent interface modules 1002(i)corresponding to the maximum fan-outs in the system 10. In that case,particular ones of the child interface modules 1001(i) and parentinterface modules 1002(i) that are not connected to child or parent datarouter nodes 22(i,j,k) may be disabled.

The switch 1003 receives data router message packets 30 from the childinterface modules 1001(i) and the parent interface modules 1002(i). Inthe case of a data router message packet from a child interface module1001(i), the switch 1003 may transmit the message packet either to achild interface module 1001(i) or to a parent interface module 1002(i).If the header field 40 of the data router message packet 30 contains avalue that corresponds to the level of the data router node 22(i,j,k),or if the AFD MODE signal indicates that the data router is inall-fall-down mode, the switch 1003 will direct the packet 30 to a childinterface module 1001(i). Otherwise, the switch directs the messagepacket 30 to a parent interface module 1002(i). Alternatively, in thecase of a data router message packet 30 from a parent interface module1002(i), the switch 1003 will transmit the message packet 30 to a childinterface module 1001(i).

The node control circuit 1004 receives selected signals from andgenerates various control signals in response. For example, the nodecontrol circuit 1004 receives binary-encoded HEIGHT (2:0) signals whichidentify the level of the data router node 22(i,j,k) and generates DECRHGT decremented height signals which are binary encoded to identify thenext lower level in the data router 15. As described above, as the datarouter nodes 22(i,j,k) transfer the data router message packets 30 downthe data router 15, the nodes decrement the value contained in theheader field 40, which identifies the level. In that case, the DECR HGTdecremented height signals are used to form the contents of the headerfield 40, as described below.

In addition, the node control circuit 1004 receives the AFD (i,j)all-fall-down (i,j) signal from the control network 14 and generates, inresponse thereto, an AFD MODE all-fall-down mode signal which controlsoperations in the child interface modules 1001(i) and the parentinterface modules 1002(i) as described below. The node control circuitalso generates an EN enable signal, which enables the data router node22(i,j,k) to operate, and P3:P0/C3:C0 DIS parent interface module/Childinterface module disable signals that disable selected ones of the childinterface modules 1001(i) and parent interface modules 1002(i).

The node control circuit 1004 also generates a set of CHILD MAP signalswhich are coupled to the child interface modules 1001(i) and parentinterface modules 1002(i) and are used to force the association of eachof the modules 1001(i) and 1002(i) with a particular the child interfacemodule 1001(i) during while the AFD MODE all-fall-down mode signal isasserted. This forces the switch 103 to couple data router messagepackets 30 received from a particular source, a particular parent orchild, to a particular child interface module 1001(i).

Finally, the node control circuit 1004 also transmits selected errorsignals, represented by an OUT ERROR signal on FIG. 11A, to thediagnostic network 16 if it detects the presence of selected errorconditions.

Each child interface module 1001(i) includes an input child circuit,generally identified by reference numeral 1006(i), and an output childcircuit generally identified by reference numeral 1007(i). The inputchild circuit 1006(i) transmits a C"i" IN FLY child "i" input fly signalto the child connected thereto, and receives C"i" IN FLIT child "i"input flit signals, comprising four signals received in parallel. TheC"i" IN FLIT signals received at successive ticks of the NODE CLK signalrepresent four-bit flits of the data router message packet 30 from thechild connected thereto.

The input child circuit 1006(i), in response to the message addressportion 31 of the message packet determines whether it is to betransmitted up the tree or down the tree defining the data router 15. Ifthe input child circuit 1006(i) determines that the data router messagepacket 30 is to be transmitted up the tree, it enables the switch 1003to direct the successive flits comprising the packet 30 to a parentinterface module 1002(i) selected by the switch 1003 at random. On theother hand, if the input child circuit 1006(i) determines that the datarouter message packet 30 is to be transmitted down the tree, itidentifies one of the child interface modules 1001(i) to which theswitch 1003 is to direct the packet 30. The switch 1003 then directs thesuccessive flits of the message packet 30 to the output child circuit1007(i) of the identified child interface module 1001(i).

Each output child circuit 1007(i) receives a C"i" OUT FLY child "i"output fly signal from the child connected thereto, and transmitsthereto C"i" OUT FLIT child "i" output flit signals, comprising foursignals transmitted in parallel. The C"i" IN FLIT child "i" input flitsignals transmitted at successive ticks of the NODE CLK signal representfour bit flits of the data router message packet 30 transmitted to thechild connected thereto.

Each parent interface module 1002(i) includes an input parent circuit,generally identified by reference numeral 1010(i), and an output parentcircuit generally identified by reference numeral 1011(i). The inputparent circuit 1010(i) transmits a P"i" IN FLY parent "i" input flysignal to the parent connected thereto, and receives P"i" IN FLIT parent"i" input flit signals, comprising four signals received in parallel. Itwill be appreciated that the C"i" OUT FLY child "i" output fly and C"i"OUT FLIT child "i" output flit signals described above in connectionwith an output child circuit 1007(i) in a data router node 22(i,j,k) atone level "i" will correspond to the P"i" IN FLY parent "i" input flyand P"i" IN FLIT parent "i" input flit signals of input parent circuits1010(i) in the child data router nodes 22(i-1,j,k) in the next lowerlevel.

The input parent circuit 1010(i), in response to the message addressportion 31 of the message packet, identifies one of the child interfacemodules 1001(i) to which the switch 1003 is to direct the packet 30. Theswitch 1003 then directs the successive flits of the message packet 30to the output child circuit 1007(i) of the identified child interfacemodule 1001(i). It will be appreciated that, in the system 10 describedherein, data router message packets 30, once they have been transmittedup the tree defined by the data router 15 to the level identified in themessage packet 30 and have started down the tree, are not thereaftertransmitted up the tree again. Accordingly, message packets 30 coupledby input parent circuits 1010(i) to switch 1003 are directed only to anoutput child circuit 1007(i), and not to an output parent circuit1011(i).

The output parent circuits 1011(i) operate in a similar manner as theoutput child circuits 1007(i). Each output parent circuit 1011(i)receives a P"i" OUT FLY parent "i" output fly signal from the parentconnected thereto, and transmits thereto P"i" OUT FLIT parent "i" outputflit signals, comprising four signals transmitted in parallel. The P"i"IN FLIT parent "i" input flit signals transmitted at successive ticks ofthe NODE CLK signal represent four bit flits of the data router messagepacket 30 transmitted to the parent connected thereto. It will beappreciated that the C"i" IN FLY child "i" input fly and C"i" IN FLITchild "i" input flit signals described above in connection with an inputchild circuit 1006(i) in a data router node 22(i,j,k) at one level "i"will correspond to the P"i" OUT FLY parent "i" output fly and P"i" OUTFLIT parent "i" output flit signals of output parent circuits 1011(i) inthe parent data router nodes 22(i+1,j,k) in the next higher level.

Thus, it will be recognized that the input child circuit 1006(i), whichreceives the C"i" IN FLIT child input flit signals representing flits ofdata router message packets 30, regulate the flow of flits thereto bymeans of the C"i" IN FLY child input fly signal. When the input childcircuit 1006(i) circuit negates the C"i" IN FLY signal while the childdata router node 22(i-1,j,k) connected thereto is transmitting a datarouter message packet 30, it stops transmitting signals, which the inputchild circuit 1006(i) receives as the C"i" IN FLIT signals, representingthe flits. The child data router node 22(i-1,j,k) effectively providesnegated C"i" IN FLIT signals.

When the input child circuit 1006(i) again asserts the C"i" IN FLYsignal, the child data router node 22(i-1,j,k) resumes transmittingsignals, which the input child circuit 1006(i) receives as the C"i" INFLIT signals, which, at successive ticks of the NODE CLK signal,represent successive flits of the packet 30. Between data router messagepackets 30, the child data router node 22(i,j,k) negates the C"i" INFLIT signals, regardless of the condition of the C"i" IN FLY signal.Accordingly, the input child circuit 1006(i) can identify the first flitof a new data router message packet 30 as the first tick of the NODE CLKsignal following the end of a previous packet at which the C"i" IN FLITsignals are not all negated.

Similarly, the input parent circuit 1010(i), which receives the P"i" INFLIT parent input flit signals representing flits of data router messagepackets 30, regulate the flow of flits thereto by means of the P"i" INFLY parent input fly signal. When the input parent circuit 1010(i)circuit negates the P"i" IN FLY signal while the parent data router node22(i+1,j,k) connected thereto is transmitting a data router messagepacket 30, the parent data router node 22(i+1,j,k) stops transmittingsignals, which the input parent circuit 1010(i) receives as the P"i" INFLIT signals, representing the flits. The parent data router node22(i+1,j,k) effectively provides negated P"i" FLIT signals.

When the input parent circuit 1010(i) again asserts the P"i" IN FLYsignal, the parent data router node 22(i+1,j,k) resumes transmittingsignals, which the input parent circuit 1010(i) receives as the P"i" INFLIT signals, which, at successive ticks of the NODE CLK signal,represent successive flits of the packet 30. Between data router messagepackets 30, the parent data router node 22(i+1,j,k) negates the P"i"FLIT signals, regardless of the condition of the P"i" IN FLY signal.Accordingly, the input parent circuit 1010(i) can identify the firstflit of a new data router message packet 30 as the first tick of theNODE CLK signal following the end of a previous packet at which the P"i"IN FLIT signals are not all negated.

The parent and child input circuits 1006(i) and 1010(i) are generallysimilar, as are the parent and child output circuits 1007(i) and1011(i). The details of child input circuit 1006(0) will be described inconnection with FIGS. 11B through 11B-3A. The details of the switch 1003will be described in connection with FIGS. 11C-1 through 11C-6. Finally,the details of output child circuit 1007(0) will be described inconnection with FIG. 11D.

2. Input Child Circuit 1006(0) i. General

FIG. 11B depicts a general block diagram of input child circuit 1006(0).With reference to FIG. 11B, the input child circuit 1006(0) includes aninterface circuit 1020, an input message control circuit 1021, an inputmessage first-in first-out buffer (FIFO) 1022, an output requestidentification FIFO 1023 and a switch input control 1024. The interfacecircuit 1020 receives a VAL FLOW valid flow signal from the inputmessage control circuit 2021, and transmits in response the C0 IN FLYinput fly signal to the parent output circuit 1011 connected thereto.

The interface circuit 1020 also receives the C0 IN FLIT input flitsignal from the same parent output circuit 1011 and couples, in responsethereto, FLIT signals to the input message control circuit 1021. The VALFLOW valid flow signal provided by the input message control circuit1021 operates as a flow control signal regulating the flow of flitsrepresented by the FLIT signals from the interface circuit 1020 to theinput message control circuit 1021. The interface circuit 1020 providesa VAL FLIT signal to the input message control circuit 1021 to indicatethat the FLIT signals represent valid flits of a data router messagepacket 30, or the binary-encoded value of zero if the interface 1020 isnot receiving a message packet 30.

The input message control circuit 1021 performs a number of operations.In particular, input message control circuit 1021 monitors the FLITsignals from the interface to detect the beginning of a data routermessage packet 30. In response to the FLIT signals representingsuccessive flits of a data router message packet 30, the input messagecontrol circuit 1021 couples IMF FLIT input message FIFO flit signalsrepresenting successive flits to the input message FIFO 1022.

In providing IMF FLIT input message FIFO flit signals to the inputmessage FIFO 1022, the input message control circuit 1021 also performssome processing on the FLIT signals representing the first two flits ofa data router message packet. In addition, the input message controlcircuit 1021 determines whether the data router message packet is to betransferred to a parent or to a child data router node 22(i,j,k), and ifto a child the particular child. The input message control circuit 1021generates OUT REQ [P,C(1:0)] output request (parent, child) signals,which it transfers to the output request identification FIFO 1023, alongwith an OIF PUSH output identification FIFO push signal. When the inputmessage control circuit 1021 asserts the OIF PUSH signal, the outputrequest identification FIFO 1023 stores the OUT REQ [P,C(1:0)] signals.

In processing the first two flits of the data router message packet 30,if the AFD MODE all-fall-down mode signal is not asserted, the inputmessage control circuit 1021 compares the binary-encoded value of theFLIT signals representing the header field 40 of the data router messagepacket 30 to the binary-encoded value of the NODE HEIGHT (2:0) signalsrepresenting the level of the data router node 22(i,j,k) in the datarouter 15. If binary-encoded value of the FLIT signals representing theheader field 40 is less than that of the NODE HEIGHT (2:0) signals, thedata router message packet 30 is to be transferred up the tree.Accordingly, the input message control circuit 1021 couples the FLITsignals representing the header field 40, as well as the FLIT signalsrepresenting the succeeding flits of the message address portion 31, asthe IMF FLIT signals to the input message FIFO 1022. In addition, theOUT REQ [P,C(1,0)] output request signals loaded onto the output requestidentification FIFO 1023 indicate that the data router message packet 30is to be transferred to the parent data router node 22(i,j,k).

On the other hand, if, while the AFD MODE all-fall-down mode signal isnot asserted, the input message control circuit 1021 determines that thebinary-encoded value of the FLIT signals representing the header field40 is the same as that represented by the NODE HEIGHT (2:0) signals, thedata router node 22(i,j,k) is to transfer the data router message packet30 down the data router 15. Accordingly, the input message controlcircuit generates IMF FLIT input message FIFO flit signals having abinary-encoded value that is one less than the binary-encoded valuerepresented by the FLIT signals representing the header field 40, whichit couples to the input message FIFO 1022 as the first flit of the datarouter message packet 30. The input message control circuit 1021 alsodetermines whether the first flit of the down path identificationportion 41 is to be eliminated, and if so does not transfer signalsrepresenting that flit to the input message FIFO 1022. In addition, theinput message control circuit 1021 generates OUT REQ [P,C(1,0)] outputrequest signals for transfer to the output request identification FIFO1023 which have a binary-encoded value that identify the child datarouter node 22(i,j,k) to which the message packet 30 is to betransmitted.

Finally, if the AFD MODE all-fall-down mode signal is asserted, theinput message control circuit 1021 transfers the FLIT signalsrepresenting the header field 40 and the down path identificationportion 41 unchanged to the input message FIFO 1022 as the IMF FLITinput message FIFO flit signals. In addition, the input message controlcircuit 1021 generates OUT REQ [P,C(1:0)] output request signals fortransfer to the output request identification FIFO 1023 which have abinary-encoded value that identifies the child data router node22(i,j,k) identified by the CHILD MAP signals.

The input message control circuit 1021 also generates a BOMbeginning-of-message signal, an NEOM near end-of-message signal and anEOM end-of-message signal, all of which it couples to the input messageFIFO 1022 along with the IMF FLIT signals representing the first flit, aflit a selected number of flits from the last, and the last flit of adata router message packet 30, respectively. The input message FIFO 1022receives and stores the BOM, NEOM and EOM signals along with thecorresponding IMF FLIT signals.

The input message control circuit 1021 generates an IMF PUSH inputmessage FIFO push signal, which it asserts to enable the input messageFIFO 1022 to receive IMF FLIT signals representing a flit, along withthe associated BOM, NEOM, and EOM signals. In addition, input messagecontrol circuit 1021 receives an IMF NR FULL input message FIFO nearlyfull signal from the input message FIFO 1022. When the IMF NR FULLsignal is asserted, the input message FIFO 1022 is nearly full. Theinput message control circuit 1021 uses the asserted or negatedcondition of the IMF NR FULL signal in controlling the assertion andnegation of the VAL FLOW signal. As noted above, the interface 1020 usesthe condition of the VAL FLOW signal in controlling the condition of theC0 IN FLY signal. Accordingly, the asserted or negated condition of C0IN FLY signal will represent the extent to which the input message FIFO1022 is able to receive and store additional flits.

Finally, the input message control circuit 1021, when it receives FLITsignals representing the check field 33 of the data router messagepacket 30, verifies that the data router message packet 30 was properlyreceived. If the input message control circuit 1021 determines that thedata router message packet 30 was properly received, it generates a newcheck value which it couples as IMF FIFO input message FIFO signals tothe input message FIFO 1022, representing the last flit of the datarouter message packet 30. If, on the other hand, the input messagecontrol circuit 1021 determines that the data router message packet 30was not properly received, it couples an error code as the IMF FIFOsignals, and also enables the node control circuit 1004 to assert an OUTERROR signal. In one embodiment, the error code generated by the inputmessage control circuit 1021 corresponds to the complement of the checkvalue that the circuit 1021 would otherwise provide.

The switch input control circuit 1024 performs several operations.First, the switch input control circuit 1024 retrieves from the outputrequest identification FIFO 1023 the buffered OUT REQ [P,C(1,0)] outputrequest signals, which it receives as ORIF OUT REQ [P,C(1:0)] bufferedoutput request signals. To initiate the retrieval, the switch inputcontrol circuit 1024 asserts an OIF POP output identification FIFO popsignal. When the output request identification FIFO 1023 is empty, itasserts an OIF EMPTY output identification FIFO empty signal.

In response to retrieved ORIF OUT REQ [P,C(1:0)] buffered output requestsignals, the switch input control circuit decodes the encoded childidentification portion to generate OUT REQ [P,C3:C0]/SW output requestto switch signals. The OUT REQ [P,C3:C0]/SW signals comprise one signalwhich, if asserted, indicate that the flits comprising the associateddata router message packet 30 are to be coupled to the parent datarouter node 22(i,j,k), and a signal associated with each child datarouter node 22(i,j,k) which, if asserted, indicate that the flits are tobe coupled to the associated child data router node 22(i,j,k). The OUTREQ [P,C3:C0]/SW signals are coupled to the switch 1003

When the switch 1003 is in a condition to couple the flits comprisingthe associated data router message packet 30 to the recipient identifiedby the OUT REQ [P,C3:C0]/SW signals, it grants the request, representedby those signals by asserting a REQ GRANTED request granted signal. Atthat point, the switch input control circuit may assert the OIF POPoutput identification FIFO pop signal and repeat the operation. If theoutput request identification FIFO 1023 is at that point asserting theOIF EMPTY output identification FIFO empty signal, the operation will bedelayed until it negates the OIF EMPTY signal.

After the switch 1003 has granted a request, the switch input controlcircuit 1024 retrieves flits representing the data router message packet30 from the input message FIFO 1022 and couples them to the switch 1003.While it is able to receive the individual flits, the switch 1003maintains a SW FLOW switch flow signal in an asserted condition. Inresponse, the switch input control circuit asserts an IMF POP inputmessage FIFO pop signal which enables the input message FIFO 1022 totransmit FLIT/FIFO buffered flit signals which, in synchrony withsuccessive ticks of the NODE CLK signal, represent successive flits ofthe data router message packet 30. The switch input control circuit, inturn, couples these signals to the switch 1003 as SW FLIT switch flitsignals, and asserts a SW FLY switch fly signal.

While the switch input control circuit 1024 is transmitting SW FLITswitch flit signals representative of successive flits of a data routermessage packet 30, it asserts the OIF POP output identification FIFO popsignal to enable the output request identification FIFO 1023 to transmitto it new ORIF OUT REQ [P,C(1:0)] signals for the next data routermessage packet 30, if any. If no additional data router message packetshave been received, the output request identification FIFO 1023maintains the OIF EMPTY output identification FIFO signal in an assertedcondition. If the OIF EMPTY signal is negated it transmits the ORIF OUTREQ [P,C(1:0)] output request signals therefor, which the switch inputcontrol circuit 1024 receives at the next tick of the NODE CLK signal,decodes and couples to the switch 1003 as described above.

The switch 1003 identifies and sets up the connection for the new datarouter message packet while the flits for the current data routermessage packet 30 are being coupled therethrough. When the switch inputcontrol circuit 1024 receives the NEOM/FIFO near end-of-message fromFIFO signal for the current data router message packet, it asserts a SWRELEASE switch release signal that enables the switch 1003 to finish theconnection for the new data router message packet 30, and the switch1003 can begin receiving SW FLIT switch flit signals for the flits ofthe data router message packet 30 immediately after all of the flits forthe current packet 30 have been coupled.

ii. Interface Circuit

FIG. 11B-1 depicts a detailed block diagram of an interface 1020 in theinput child circuit 1006(0). With reference to FIG. 11B-1, the interface1020 includes a receiver 1030 that receives the C0 IN FLIT input flitsignals at a data input terminal, and latches them at each tick of theNODE CLK signal. The data output terminal of latch 1030 provides theFLIT signals, which are coupled to the input message control circuit1021.

A latch driver 1031 transmits the C0 IN FLY input fly signal. The VALFLOW valid flow signal from the input message control circuit 1021 isreceived at one input terminal of a multiplexer 1032. The multiplexer'sother input terminal is maintained at a negated signal level. Themultiplexer 1032 is controlled by a C0 NOT DIS not disabled signals,which represents the complement of the C0 DIS disable signal from nodecontrol circuit 1004 (FIG. 5). As noted above, the node control circuit1004 asserts the C0 DIS disable signal to disable the child interfacemodule 1001(0).

If the C0 DIS signal is asserted, the multiplexer 1032 couples a negatedlevel signal as a RAW FLOW signal to the data input terminal of thelatch driver 1031. The negated-level RAW FLOW signal, which the latchdriver 1031 latches at each tick of the NODE CLK signal, maintains theC0 IN FLY input fly signal at a negated condition.

In addition, the RAW FLOW raw flow signal is coupled to the inputterminal of a delay line 1033. The delay line couples the delayed RAWFLOW signal as the VAL FLIT valid flit signal to the input messagecontrol circuit 1021. While the RAW FLOW signal is maintained in anegated condition, the VAL FLIT signal is also negated. If, however, thecondition of the RAW FLOW signal changes from the asserted state to thenegated state, the delay line 1033 delays provides a delay in atransition of the VAL FLIT signal from an asserted state to a negatedstate.

The amount of delay provided by delay line 1033 is on the order of thepropagation time for the C0 IN FLY input fly signal from the child inputinterface 1020 to the output interface of the child data router node22(i,j,k) or injector port 225 of the leaf 21 connected thereto, sincethat circuit may have transmitted signals, which will be received as C0IN FLIT input flit signals representing several flits before it receivesthe negated C0 IN FLY signal. These flits will be received by thereceiver 1030 and coupled to the input message control circuit 1021. Thedelayed negation of the VAL FLIT valid flit signal enables the inputmessage control circuit 1021 to receive these flits from the receiver1030 and couple them to the input message FIFO 1022. The delay providedby delay line 1033 ensures that the negation of the VAL FLIT signal, asreceived by the input message control circuit 1021, is synchronous withthe provision thereto of FLIT signals by receiver 1030 representing athe last flit provided by the child data router node 22(i,j,k) orinjector port 223 of the leaf 22 connected thereto. It will beappreciated that the input message FIFO 1022 will enable the inputmessage control 1021 to negate the VAL FLOW signal sufficiently ahead ofits becoming full to enable it to receive and store these additionalflits.

On the other hand, if the C0 DIS signal is negated, the multiplexer 1032couples the VAL FLOW signal from the input message control circuit asthe RAW FLOW signal. The RAW FLOW signal, accordingly, reflects theability of the input message FIFO 1022 to accept and store additionalflits. If the RAW FLOW signal is negated, indicating that the inputmessage FIFO 1022 is nearly full, the VAL FLOW signal is will also benegated. As above, the negated VAL FLOW signal will be latched by driver1031 in response to the NODE CLK signal, which will drive the C0 IN FLYinput fly signal to the data router node 22(i,j,k) to a negated level todisable the child data router node 22(i,j,k) or leaf 21 connectedthereto from transmitting flits.

If, however, the input message FIFO 1022 is able to receive additionalflits, the input message control circuit 1021 asserts the VAL FLOW validflow signal. If the C0 DIS disable signal is negated, the multiplexer1032 couples the asserted level VAL FLOW valid flow signal as anasserted level RAW FLOW signal to the input terminal of driver 1031.Driver 1031, in turn, latches the asserted RAW FLOW signal and drives itas an asserted C0 IN FLY input fly signal at the next tick of the NODECLK signal.

The delay line 1033 also receives the asserted RAW FLOW signal andasserts the VAL FLIT valid flit signal. In a transition of the VAL FLITsignal from a negated condition to an asserted condition, the transitionin the VAL FLIT signal is also delayed by the delay line 1033. As above,the delay is provided to permit the child data router node 22(i,j,k) orinjector port 223 of the leaf 21 connected thereto to receive theasserted C0 IN FLY input fly signal and begin transmitting signals,which will be received at receiver 1030 as the C0 IN FLIT signals,representing flits. The delay provided by delay line 1033 ensures thatthe assertion of the VAL FLIT signal, as received by the input messagecontrol circuit 1021, is synchronous with the provision thereto of FLITsignals by receiver 1030 representing a flit.

iii. Input Message Control Circuit

FIG. 11B-2 depicts a general block diagram of the input message controlcircuit 1021 (FIG. 11B). With reference to FIG. 11B-2, the input messagecontrol circuit 1021 includes a fit flow control circuit 1040, an inputcontrol circuit 1041, a down path identification flit counter 1042, adata flit counter 1043 and a checksum check/generator circuit 1044. Theflit flow control circuit 1040, which is depicted in greater detail inFIGS. 11B-2B and 11B-2C, receives the FLIT signals from the interface1020 and generates, under control of the input control circuit 1041, IMFFLIT input message FIFO flit signals and the BOM beginning-of-messagesignal, NEOM near end-of-message signal, and EOM end-of-message signalfor storage in the input message FIFO 1022. In addition, the flit flowcontrol circuit 1040 generates the OUT REQ [P,C(1:0)] output requestsignals for storage in the output request identification FIFO 1023. Theflit flow control circuit 1040 also generates the VAL FLOW signal usedby the interface circuit 1020 and receives the VAL FLIT valid flitsignal which controls its use of the FLIT signals.

The flit flow control circuit also provides several signals to the inputcontrol circuit 1041. The input control circuit 1041 is essentially astate machine, that operates in a plurality of states as depicted onFIG. 11B-2A. The input control circuit 1041 provides several signals tothe flit flow control circuit 1040 that effectively identify variousstates in receipt of a data router message packet 30. In particular, theinput control circuit 1041 provides a MSG INIT message initializationsignal that enables the flit flow control circuit 1040 to identify thefirst flit of a data router message packet. If the MSG INIT signal isasserted, and if the VAL FLIT signal is asserted, the first flit of adata router message packet 30 is indicated by the FLIT signals receivedby the flit flow control circuit 1040 having a non-zero binary-encodedvalue.

The asserted MSG INIT message initialization signal is also coupled tothe down path identification flit counter 1042 to enable it to load theFLIT signals. It will be appreciated that the input message controlcircuit 1041 will maintain the MSG INIT signal in an asserted conditionuntil the FLIT signals represent the header field 41 of a data routermessage packet 30, and while the MSG INIT signal is asserted the counter1042 will continually load the FLIT signals.

As described above, if a data router node 22(i,j,k) receives a datarouter message packet, traversing the data router 15 down the tree, thenode may discard flits representing the down path identification portion41 as they are used. The input control circuit 1041 generates a DROP DNFLIT drop down flit signal which, if asserted, enables the flit flowcontrol circuit 1040 to discard the first flit of the down pathidentification portion 41. The input control circuit 1041 uses aHDR=header equal signal, the low-order NODE HEIGHT(0) height signal andthe AFD MODE all-fall-down mode signal to determine whether to assertthe DROP DN FLIT drop down flit signal. If the flit flow control circuit1040 is asserting the HDR=signal, the data router message packet 30 isbeing transmitted down the data router tree. A data router node22(i,j,k) drops a flit in the down path identification portion 41 onlyin the odd-numbered levels (i), which is indicated by the low-order NODEHEIGHT(0) signal. Finally, the data router nodes 22(i,j,k) do not dripflits in the down path identification portion 41 if data router 15 is inall-fall-down mode, as indicated by the assertion of the AFD MODEall-fall-down mode signal.

Finally, the input control circuit 1041 also asserts a MSG LEN INmessage length in signal, a TAG OR DATA signal and a CHECK signalcontemporaneously with FLIT signals representing the corresponding flitsof the data router message packet. In generating these signals, theinput control circuit uses signals from the down path identificationflit counter 1042 and the data flit counter 1043.

The flit flow control circuit 1040 also provides the received FLITsignals to the down path identification flit counter 1042, the data flitcounter 1043 and the checksum check/generator circuit 1044. The downpath identification flit counter 1042, which will, be described ingreater detail below in connection with FIG. 11B-2E, receives the FLITsignals representative of the header field 40 (FIG. 3) of a data routermessage packet 30. The counter 1042 iteratively decrements in responseto successive ticks of the NODE CLK signal while the VAL FLIT valid flitsignal is asserted. During this time, the FLIT signals represent thedown path identification portion 41 of the data router message packet 30being received. When the value of counter 1042 decrements to zero, itasserts a DNF END down flit end signal, which indicates that the entiredown flit identification portion 41 has been received.

Similarly, the data flit counter 1043, which will be described below inconnection with FIG. 11B-2D, receives the FLIT signals representative ofthe message length field 34 of a data router message packet 30. Thecounter 1042 iteratively decrements in response to successive ticks ofthe NODE CLK signal while the VAL FLIT valid flit signal is asserted.During this time, the FLIT signals represent the message data portion 32of the data router message packet 30 being received. When the value ofcounter 1043 decrements to nearly zero, it asserts a NR DATA END signal,which the flit flow control circuit uses in generating the NEOM nearend-of-message signal. When counter 1043 decrements to zero, it assertsa DATA END data end signal, which indicates that the entire message dataportion 32 has been received. The flit flow control circuit uses theDATA END signal in generating the EOM end-of-message signal.

The checksum check/generator circuit 1044 receives the FLIT signals andverifies the checksum of the FLIT signals representative of check field33. The checksum check/generator circuit 1044 is reset in response tothe BOM beginning-of-message signal provided by the flit flow controlcircuit 1040. In response to FLIT signals representative of successiveflits of a data router message packet 30, the checksum check/generatorcircuit 1044 generates NEW CHECKSUM signals. When the input controlcircuit 1041 asserts the CHECK signal, the circuit 1044 generates aCHECK OK signal whose condition indicates whether the checksum generatedby the checksum check/generator circuit matches the value in the checkfield 33 of the data router message packet. The flit flow controlcircuit 1040 uses these signals in generating the IMF FLIT input messageFIFO flit signals representing the check field 33 of the data routermessage packet for storage in the input message FIFO 1022.

The flit flow control circuit 1040 will be described in connection withFIGS. 11B-2B and 11B-2C, and in connection with FIG. 11B-2A, whichdepicts a state transition diagram for the input control circuit 1041.FIG. 11B-2B depicts a portion of the flit flow control circuit thatgenerates signals for the input message FIFO 1022, and FIG. 11-2Cdepicts a portion that generates signals for the output requestidentification FIFO 1023.

With reference to FIG. 11B-2B, the flit flow control circuit 1040receives the FLIT signals at an OR gate 1050. If at least one of theflit signals is asserted, the OR gate 1050 is energized, which enablesone input of an AND gate 1051. If the VAL FLIT valid flit signal is alsoasserted, AND gate 1051 is energized to assert a NOT ZERO signal. If theinput control circuit 1041 is asserting the MSG INIT messageinitialization signal, and if the node control circuit 1004 (FIG. 11A)is asserting the EN enable signal, the assertion of the NOT ZERO signal,while the VAL FLIT signal is asserted, energizes the AND gate 1052,enabling it to assert a FIRST FLIT signal. A driver 1053 receives theasserted FIRST FLIT signal and asserts BOM beginning-of-message signal.

The NOT ZERO signal is also coupled to an inverter 1054, which generatesa ZERO signal. If the ZERO signal is asserted, either the VAL FLITsignal is negated, which disables AND gate 1051, or the OR gate 1050 maybe de-energized. The OR gate 1050 is de-energized if all of the FLITsignals are negated.

With reference to FIG. 11B-2A, as noted above, the input control circuit1041 is initially asserting the MSG INIT message initialization signal.At that point, the circuit 1041 is in a message initialization state,indicated by the box labelled "MSG INIT." The input control circuitremains in that state if the VAL FLIT valid flit or EN enable signalsare not asserted, or of the inverter 1054 is asserting the ZERO signal.However, while the input control circuit 1041 is in the messageinitialization state, in response to the coincidence of the assertion ofthe VAL FLIT and EN signals, and the assertion by AND gate 1051 of theNOT ZERO signal resulting in the negation of the ZERO signal, the inputcontrol circuit 1041 sequences to one of three other states, asdescribed below.

Returning to FIG. 11B-2B, the FLIT signals are also coupled to the inputterminal of a gated driver 1055 and to an input terminal of a comparator1056. The gated driver 1055 controls the coupling of FLIT signals to theinput message FIFO 1022 in response to a FLIT/FIFO EN flit to FIFOenable signal from an AND gate 1057. The comparator 1056 receives theFLIT signals and the NODE HEIGHT (2:0) signals identifying the level "i"of the data router node 22(i,j,k) in the tree of data router 15. If theAND gate 1052 is asserting the FIRST FLIT signal, which occurs when theFLIT signals represent the header field 40 of the data router messagepacket 30, the comparator 1056 is enabled to compare the binary-encodedvalue of the NODE HEIGHT (2:0) signals to that of the FLIT signals.

If the comparator 1056 determines that the binary-encoded value of theFLIT signals is greater than that of the NODE HEIGHT (2:0) signals, itasserts a HDR<header less than node height signal. It will beappreciated that, with the binary-encoded value of the FLIT signals inthis condition, the data router message packet 30 is to be transmittedup the tree defined by the data router 15, unless the AFD MODEall-fall-down mode signal is asserted. On the other hand, if thecomparator 1056 determines that the binary-encoded value of the FLITsignals is less than or equal to that of the NODE HEIGHT (2:0) signals,which will occur if the data router message packet 30 is to betransmitted down the tree defined by the data router 15, it asserts aHDR=header equals node height signal.

If the comparator 1056 is asserting the HDR=header equals node heightsignal, and if the AFD MODE all-fall-down mode signal is negated and theinput control circuit 1041 is asserting the MSG INIT messageinitialization signal, an AND gate 1060 is de-energized enabling it toassert a NEW HDR EN new header enable signal. It will be appreciatedthat this occurs when the FLIT signals represent the header field 40 ofa data router message packet 30 to be transmitted down the treerepresented by the data router 15 while it is not in all-fall-down mode.As noted above, the contents of the header field 40 as transmitted by adata router node 22(i,j,k) have a binary-encoded value corresponding tothat as received, decremented by one.

To provide the new contents for header field 40, the asserted NEW HDR ENnew header enable signal, in turn, enables a gated driver 1061 tocoupled DECR HEIGHT decremented height signals as NEW HDR new headersignals onto a bus 1062. The DECR HEIGHT decremented height signals areprovided by a decrementation circuit 1063, which receives the NODEHEIGHT (2:0) signals and generates DECR HEIGHT Signals whosebinary-encoded value is one less than that of the he NODE HEIGHT (2:0)signals node height. Accordingly, the NEW HDR signals have abinary-encoded value one less than that of the NODE HEIGHT (2:0) signal.

If the comparator 1056 is not asserting the HDR=header equals nodeheight signal, indicating that the data router message packet 30 is tobe transferred up the tree defining data router 15, an AND gate 1060 isdisabled. AND gate 1060 is also disabled if the AFD MODE all-fall-downmode signal is asserted indicating that the data router 15 is inall-fall-down mode, and if the MSG INIT message initialization signal isnegated, indicating that the FLIT signals represent flits of a datarouter message packet 30 other than the header 40. In any of thesecases, an inverter 1070 enables one input terminal of AND gate 1057. Ifthe input control circuit 1041 is not asserting the CHECK signal, aninverter 1071 enables the second input terminal of AND gate 1057,enabling it to assert the FLIT/FIFO EN flit to FIFO enable signal. Theasserted FLIT/FIFO EN signal enables the AND gate 1055 to couple theFLIT signals as FLIT TO FIFO signals onto bus 1062. Thus, the gate 1055couples the FLIT signals to the bus 1062 if:

(i) the FLIT signals represent the header field 40 and the data routermessage packet 30 is to be transferred up the tree defining data router15;

(ii) the FLIT signals represent the header field 40 and the AFD MODEall-fall-down mode signal is asserted; and

(iii) the FLIT signals represent fields other than the header field 40or the check field 33.

While the FLIT signals represent the header field 40, the asserted MSGINIT message initialization and NOT ZERO signals energize an AND gate1064 to assert a BEG MSG begin message signal. The asserted BEG MSGsignal energizes an OR gate 1065, which, in turn, enables one inputterminal of an AND gate 1066. Since the VAL FLIT valid flit signal isasserted, the second input terminal of AND gate 1066 is also enabled,energizing the AND gate and enabling it to assert the IMF PUSH inputmessage FIFO push signal. The asserted IMF PUSH signal enables the inputmessage FIFO 1022 to store the IMF FLIT input message FIFO flit signals,which, as noted above, have a binary-encoded value corresponding to thebinary-encoded value of the NODE HEIGHT (2:0) signals decremented byone.

Thereafter, the operations of the flit flow control circuit will dependon the particular state, following the message initialization state, towhich the input control circuit 1041 sequences (FIG. 11B-2A). As notedabove, the input control circuit 1041 may sequence from the messageinitialization state to one of three states. In two states, namely, akeep down path identification flit state identified by the box labelled"KEEP DN PATH ID FLIT," and a receive down path identification flitsstate identified by the box labelled "RCV DN PATH ID FLITS," the inputcontrol circuit 1041 enables the flit flow control circuit 1040 toreceive FLIT signals representing the all of the successive flits of thedown path identification portion 41 of the data router message packet30. In that operation, the input control circuit 1041 negates the MSGINIT message initialization signal and maintains the DROP DN FLIT dropdown path identification flit signal in a negated condition.

In the third state, identified as a drop down path identification flitstate represented by the box labelled "DROP DN PATH ID FLIT," the inputcontrol circuit 1041 enables the flit flow control circuit 1040 to dropthe first flit of the down path identification portion 41. In thatstate, the input control circuit 1041 asserts the DROP DN FLIT drop downpath identification flit signal, and also negates the MSG INIT messageinitialization signal.

The input control circuit 1041 sequences from the message initializationstage in response to the coincidence of several conditions representedby signals having selected states. If the VAL FLIT valid flit, ENenable, and NOT ZERO signals are asserted, and if either the AFD MODEall-fall-down mode signal is asserted or the HDR=header equals nodeheight signal is negated, the input control circuit 1041 sequences tothe receive down path identification flits state. Thus, if the datarouter 15 is in all-fall-down mode, or if the flit flow control circuit1040 determines, while the input control circuit is in the messageinitialization state, that the data router message packet 30 is to betransferred up the tree defining data router 15, the input controlcircuit 1041 sequences to the receive down path identification state.

On the other hand, if the VAL FLIT valid flit, EN enable, and NOT ZEROsignals are asserted, and if the HDR=header equals node height signal isasserted while the NODE HEIGHT (0) and AFD MODE all-fall-down modesignals are -. negated, the input control circuit 1041 sequences to thekeep down path identification flit state. Thus, if (a) the data routeris not in all-fall-down mode, (b) the level "i" of the data router node22(i,j,k) is an even number, and (c) the data router message packet isto be transferred down the tree defining data router 15, the inputcontrol circuit 1041 sequences to the keep down path identification flitstate.

Finally, if the VAL FLIT valid flit, EN enable, and NOT ZERO signals areasserted, and if the HDR=header equals node height and NODE HEIGHT (0)signals are asserted when the AFD MODE all-fall-down mode signal isnegated, the input control circuit 1041 sequences to the drop down pathidentification flit state. Thus, if (a) the data router is not inall-fall-down mode, (b) the level "i" of the data router node 22(i,j,k)is an odd number, and (c) the data router message packet is to betransferred down the tree defining data router 15, the input controlcircuit 1041 sequences to the drop down path identification flit state.As noted above, in that state the input control circuit 1041 asserts theDROP DN FLIT drop down flit signal.

Returning to FIG. 11B-2B, when the input control circuit 1041 sequencesfrom the message initialization state it negates the MSG INIT messageinitialization signal. The negated MSG INIT signal is complemented byinverter 1070 to enable one input terminal of AND gate 1057. In any ofthe states immediately following the message initialization state, theinput control circuit 1041 also negates the CHECK signal, which iscomplemented by inverter 1071. In that condition, the AND gate 1057 isenergized, enabling it to assert the FLIT/FIFO EN flit to FIFO enablesignal, which enabled gated driver 1055 to couple the FLIT signals asFLIT TO FIFO signals onto bus 1062.

The negated MSG INIT signal also disables the AND gate 1064, causing itto negate the BEG MSG begin message signal, which, in turn, disables oneinput terminal of the OR gate 1065. The negated MSG INIT messageinitialization signal, on the other hand, is complemented by an inverter1072 to enable one input of an AND gate 1073. If the DROP DN FLIT signalis negated, which occurs if the input message control sequences to thekeep down path identification flit state or the receive down pathidentification flits state, an inverter 1074 enables the second inputterminal of AND gate 1073, thereby energizing the AND gate to assert aCONT MSG continue message signal. The asserted CONT MSG signal maintainsthe OR gate 1065 in an energized condition, enabling, in turn, AND gate1066 to maintain the IMF PUSH input message FIFO push signal in anasserted condition while the VAL FLIT signal remains asserted. Thus, ifthe input control circuit 1041 sequences from the message initializationstate to either the keep down path identification flit state or thereceive down path identification flits state, it enables the flit flowcontrol circuit 1040 to maintain the IMF PUSH input message FIFO pushsignal at an asserted level while FLIT TO FIFO signals representing boththe header field 40 and the first flit of the down path identificationportion 41 are coupled onto bus 1062.

On the other hand, if the input control circuit 1041 sequences from themessage initialization state to the drop down path identification flitstate, and thereby asserts the DROP DN FLIT drop down flit signal, theinverter 1074 disables the AND gate 1073 to negate the CONT MSG continuemessage signal. Since the AND gate 1064 is also negating the BEG MSGbegin message signal, the OR gate 1065 is de-energized, disabling ANDgate 1066 to thereby negate the IMF PUSH input message FIFO push signal.The AND gate 1055 is at that point coupling the FLIT signals as FLIT TOFIFO signals onto bus 1062 representing the first flit of the down pathidentification portion 41. However, since the IMF PUSH signal isnegated, the input message FIFO 1022 is disabled from storing thesignals. Accordingly, the asserted DROP DN FLIT drop down flit signalcauses the first flit of the down flit identification portion 41 to bedropped from the data router message packet 30 to be transferred.

The negation of the MSG INIT message initialization signal by the inputcontrol circuit 1041 also inhibits the down path identification flitcounter 1042 from continuing to load the FLIT signals. Thereafter, whilethe VAL FLIT valid flit signal is asserted, the counter 1042 is enabledto decrement in response to each tick of the NODE CLK signal. As notedabove, while the VAL FLIT signal is asserted, the FLIT signals atsuccessive ticks of the NODE CLK signal represent successive flits ofthe data router message packet 30. When the MSG INIT messageinitialization signal is negated, the FLIT signals loaded by the downpath identification flit counter 1042 represent the header field 40 ofthe data router message packet 30 being transferred, which, as notedabove, identifies the number of flits in the down path identificationportion 41.

Returning to FIG. 11B-2A, the input control circuit 1041 stays in thedrop down path identification flit state or the keep down pathidentification flit state while the flit flow control circuit 1040receives only one flit, namely, the first flit of the down pathidentification portion. Thereafter, the input control circuit 1041sequences to either the receive down path identification flits state ora receive message length in state, the latter being represented by thebox labelled "RCV MSG LEN IN." If, with the input control circuit 1041being in the drop down flit identification flit state or in the keepdown path identification flit state, the down path identification flitcounter 1042 is not asserting the DNF END down flit end signal, theinput control circuit 1041 sequences to the receive down pathidentification flits state. In that state, the input control circuit1041 continues to maintain all of its output signals, including the DROPDN FLIT signal, in a negated condition. As described above, while theVAL FLIT signal is asserted, the negated DROP DN FLIT signal maintainsthe AND gate 1066 (FIG. 11B-2B) energized to maintain the IMF PUSH inputmessage FIFO push signal in an asserted condition.

If, on the other hand, when the input control circuit 1041 is in eitherthe drop down path identification flit state or the keep down pathidentification flit state and the down path identification flit counter1042 counts out and asserts the DNF END signal, it (that is, the inputcontrol circuit 1041) will sequence to the receive message length instate. That will occur if the down path identification portion 41 of thedata router message packet 30 being received has a length of one flit.If the input control circuit 1041 is in the receive down pathidentification flits state, it will also sequence to the receive messagelength in state when the down path identification flit counter 1042counts out and asserts the DNF END signal. This will occur when the downpath identification portion 41 of the data router message packet 30 thatis being received has a length greater than one flit. In either case,when the input control circuit is in the receive message length instate, it asserts a MSG LEN IN message length in signal.

The input control circuit 1041 asserts the MSG LEN IN message length insignal when the FLIT signals represent the message length field 34 ofthe data router message packet 30. The asserted MSG LEN IN signalenables the data flit counter 1043 to load the FLIT signals. The MSG LENIN signal also enables one input terminal of an AND gate 1075. If theFLIT signals have a binary-encoded value of zero, which will beindicated if the inverter 1054 is asserting the ZERO signal, the messagedata portion 32 has no data flits 36. In that case, the second inputterminal of the AND gate 1075 is also enabled, energizing the AND gate,enabling it to assert a NO DATA signal. The asserted NO DATA signalenergizes OR gate 1076, enabling it to assert the NEOM nearend-of-message signal, Which is coupled to the input message FIFO 1022as described above.

On the other hand, if the FLIT signals do not have a binary-encodedvalue of zero, the data flit counter 1043 will decrement, while the VALFLIT valid flit signal is asserted, in response to successive ticks ofthe NODE CLK signal, and will assert the NR DATA END near data endsignal a predetermined number of flits prior to the last data flit 36,and the DATA END signal contemporaneously with the last data flit 36 inthe data router message packet 30 being received.

With the input control circuit 1041 in the receive message length instate, if the VAL FLIT signal valid flit signal is asserted the inputcontrol circuit 1041 sequences to a receive tag or data state. In thatstate, the input control circuit 1041 asserts a TAG OR DATA signal. Theinput control circuit maintains the TAG OR DATA signal in the assertedcondition while the FLIT signals represent the tag field 35 or dataflits 36 of the received data router message packet 30. The asserted TAGOR DATA signal enables one input terminal of an AND gate 1077. When thedata flit counter 1043 asserts the NR DATA END near data end signal, thesecond input terminal of AND gate 1077 is enabled, thereby energizingthe AND gate 1077 to assert a NR END near end signal. The asserted NREND signal energizes the OR gate 1076 to assert the NEOM near end ofmessage signal.

Thereafter, the data flit counter 1043 decrements again in response toreceipt of the next flit of the data router message packet 30. At thatpoint, the counter negates the NR DATA END near data end signal, causingthe AND gate 1077 to be de-energized and thereby negating the NR ENDnear end signal. The negation NR END signal, in turn, de-energizes theOR gate 1076 to negate the NEOM near end-of-message signal.

While the input control circuit 1041 is in the receive tag or datastate, if the VAL FLIT valid flit signal is asserted the flit flowcontrol circuit 1040 maintains the IMF PUSH input message FIFO pushsignal in the asserted state to enable the input message FIFO 1022 toreceive and store the successive flits of the data router message packet30. While the input control circuit 1041 is in the receive tag or datastate, it maintains the MSG INIT message initialization and DROP DN FLITdrop down flit signals in the negated condition. Accordingly, inverters1072 and 1074 maintain the AND gate 1073 in the energized condition,which, in turn, maintain OR gate 1065 energized to enable one inputterminal of AND gate 1066. While the VAL FLIT valid flit signal isasserted, the AND gate 1066 maintains the IMF PUSH signal asserted, asindicated above.

As shown on FIG. 11B-2A, when the data flit counter 1044 asserts theDATA END signal, the input control circuit 1041 sequences to a checkstate. During the check state, the input control circuit 1041 enablesthe flit flow control circuit 1040 to couple IMF FLIT input message FIFOflit signals corresponding to either the true or complement of the NEWCHECKSUM signals generated by the checksum check/generator circuit 1044as the checksum field 33 of the data router message packet 3. If thechecksum check/generator circuit 1044, while receiving the successiveflits of the data router message packet 30 being received, computes achecksum that corresponds to the checksum in the checksum field 33, theflit flow control circuit 1040 couples the true of the NEW CHECKSUMsignals as the IMF FLIT signals. On the other hand, if the checksumcheck/generator circuit 1044 computes a checksum that differs from thechecksum in the checksum field 33, the flit flow control circuit 1040couples that the complement of the NEW CHECKSUM signals as the IMF FLITsignals.

By providing the complement of the checksum signals in the checksumfield 33 upon detection of an error by the checksum check/generatorcircuit 1044, the flit flow control circuit 1040 enhances the likelihoodthat the error indication will propagate through subsequent data routernodes 22(i,j,k) in the path from the source leaf 21(x) to thedestination leaf 21(y). Since all of the signals in the checksum field33 are complemented, the next data router node 22(i,j,k) to receive thedata router message packet 30 will likely detect an error indication,and will couple the complement of the checksum signals computed by theChecksum check/generator circuit 1044 at that node to the next datarouter node 22(i,j,k), and so on. In particular, since all of thesignals in the checksum field 33 are complemented, single-bit errors inone or several of the signals in the transmission through subsequentdata router nodes 22(i,j,k) is unlikely to result in masking of theerror condition in connection with the data router message packet 30.

To accomplish this, in the check state, the input control circuit 1041negates the TAG OR DATA signal and asserts a CHECK signal. The assertedCHECK signal enables the checksum check/generator circuit 1044 (FIG.11B-2) to transmit the checksum value generated thereby as NEW CHECKSUMsignals, and to transmit a CHECK OK signal indicating whether the flitflow control circuit 1040 properly received the data router messagepacket 30. If the CHECK OK signal is asserted, the flit flow controlcircuit 1040 properly received the data router message packet 30, and ifthe signal is negated the flit flow control circuit 1040 did notproperly receive the data router message packet 30.

The CHECK, NEW CHECKSUM and CHECK OK signals are also coupled to theflit flow control circuit 1040. As shown on FIG. 11B-2B, the assertedCHECK signal is complemented by inverter 1071 to disable AND gate 1057,thereby disabling gated driver 1055 from coupling the FLIT signals thenbeing received as FLIT TO FIFO signals onto bus 1062. It will beappreciated that the FLIT signals at this point correspond to theCHECKSUM field 33 of the data router message packet 30 being received.The CHECK signal also controls two gated drivers 1080 and 1081, whichare also controlled by the true and complement, respectively, of theCHECK OK signal from checksum check/generator circuit 1044. The gateddriver 1080 also receives the true of the NEW CHECKSUM signals, andgates them as NEW CHECKSUM TO FIFO signals onto bus 1062 if both theCHECK and CHECK OK signals are asserted.

An inverter 1082 receives the NEW CHECKSUM signals and generatescomplements of the respective signals, which are coupled to inputterminals of gated driver 1081. If the CHECK OK signal is negated,indicating that the checksum check/generator circuit 1044 detected anerror in the received data router message packet 30, an inverter 1083enables a respective input terminal of gated driver 1081. If the CHECKsignal is also asserted, the gated driver 1081 couples the complementsof the NEW CHECKSUM signals provided by inverter 1082 as BAD CHECKSUM TOFIFO signals onto bus 1062.

In either case, since at this point the input control circuit ismaintaining the MSG INIT message initialization and DROP DN FLIT dropdown flit signals in a negated condition, the AND gate 1073 and OR gate1065 are energized. If the VAL FLIT signal is asserted, AND gate 1066remains energized to assert the IMF PUSH input message FIFO push signal,enabling the input message FIFO 1022 to load the signals on bus 1062 asthe new checksum field 33 of the data router message packet 30. If theCHECK OK signal is asserted, indicating that the data router messagepacket 30 as received was properly received, the input message FIFO 1022loads the NEW CHECKSUM TO FIFO signals as the checksum field 33.However, if the CHECK OK signal is negated, indicating that the datarouter message packet 30 as received was not properly received, theinput message FIFO 1022 loads the BAD CHECKSUM TO FIFO signals as thechecksum field 33.

The CHECK signal is also coupled to a driver 1084 which provides the EOMend-of-message signal. When the CHECK signal is asserted, the driver1084 asserts the EOM signal, which is loaded along with the IMF FLITinput message FIFO flit signals on bus 1062.

Returning to FIG. 11B-2A, when the input control circuit 1041 is in theCHECK state, if the VAL FLIT signal is asserted it sequences to themessage initialization state at the next tick of the NODE CLK signal.Thus, the input control circuit 1041 remains in the check state whilethe flit flow control circuit 1040 receives FLIT signals representingone flit, namely the flit representing checksum field 33. When the inputcontrol circuit 1041 leaves the check state, it negates the CHECKsignal, which disables gated drivers 1080 and 1081 (FIG. 11B-2B). Inaddition, the negated CHECK signal is complemented by inverter 1071 toenable one input terminal of AND gate 1057, to allow the AND gate 1057to be thereafter controlled by the complemented NEW HDR EN new headerenable signal as provided by inverter 1070.

As described above, when the input control circuit 1041 is in themessage initialization state, it asserts the MSG INIT messageinitialization signal, which enables respective input terminals of ANDgates 1052, 1060 and 1064, and through inverter 1072 disables an inputterminal of AND gate 1073. Thus, the flit flow control circuit 1040 isin condition to begin receiving flits for a new data router messagepacket 30.

The flit flow control circuit 1040 also provides the VAL FLOW valid flowsignal to the interface 1020, which the interface uses to control thecondition of the C0 IN FLY input fly signal. With reference to FIG.11B-2B, the flit flow control circuit 1040 includes an inverter 1085,which receives the IMF NR FULL input message FIFO nearly full signalfrom the input message FIFO 1022 and transmits the complement as the VALFLOW signal. Thus, the VAL FLOW signal reflects the extent to which theinput message FIFO 1022 has been filled.

As noted above, the flit flow control circuit 1040 also provides the OUTREQ [P,C(1:0)] output request signal and OIF PUSH output identificationFIFO push signal to output request identification FIFO 1023. Thecircuitry for this is depicted on FIG. 11B-2C. With reference to FIG.11B-2C, the circuitry includes a parent request generator portion 1090,a child request generator portion 1091 and a push signal generatorportion 1092. The parent request generator portion 1090 includes an ANDgate 1093 that receives a CHILD PORT signal which is asserted if thecircuit is in an input child interface 1006(i), and the complement ofthe AFD MODE all-fall-down mode signal as complemented by an inverter1094. If the CHILD PORT signal is asserted and the AFD MODEall-fall-down mode signal is negated, if HDR<header value less than nodeheight signal from comparator. 1056 is asserted, indicating that thedata router message packet 30 is to be transmitted up the tree definingdata router 15, the AND gate 1093 is energized to assert a GO UP signal.

The GO UP signal is coupled to one data input terminal of a multiplexer1095. If the VAL FLIT signal is asserted, multiplexer 1095 couples theGO UP signal, now asserted, to the data input terminal of a flip-flop1096, which is set in response to the next tick of the NODE CLOCKsignal. The set flip-flop 1096 asserts the OUT REQ [P] output request(parent) signal, which forms one of the OUT REQ [P,C(1:0)] outputrequest signals. If the VAL FLIT signal is negated with the flip-flop1096 in that condition, multiplexer 1095 is enabled to couple the OUTREQ [P] signal to the data input terminal of the flip-flop 1096, so thatthe flip-flop 1096 will remain set in response to subsequent ticks ofthe NODE CLK signal.

If (a) the HDR<header value less than node height signal from comparator1056 is negated, or (b) the CHILD PORT signal is negated indicating thatthe circuit is in an input parent interface 1010(i), or (c) the AFD MODEall-fall-down mode signal is asserted, indicating that the data router15 is in all-fall-down mode, the data router node 22(i,j,k) is totransfer the data router message packet 30 being received down the treedefining the data router 15. In that case, the AND gate 1093 isde-energized to negate the GO UP signal. If the VAL FLIT valid flitsignal is asserted, multiplexer 1095 couples the negated GO UP signal tothe data input terminal of flip-flop, which is clear in response to thenext tick of the NODE CLK signal, to, in turn, negate the OUT REQ [P]output request (parent) signal. If the VAL FLIT signal is negated withthe flip-flop 1096 in that condition, multiplexer 1095 is enabled tocouple the OUT REQ [P] signal to the data input terminal of theflip-flop 1096, so that the flip-flop 1096 will remain clear in responseto subsequent ticks of the NODE CLK signal.

The child request generator portion 1091 includes a gated driver 1100that receives the FLIT signals from the interface 1020 and the VAL FLITvalid flit signal. If the VAL FLIT signal is asserted, the gated driver1100 couples the FLIT signals as GATED FLIT signals to input terminalsof a multiplexer 1101. As noted above, the FLIT signals comprise foursignals in parallel. The high-order GATED FLIT signals, identified asGATED FLIT (3:2) signals, are coupled to one set of input terminals ofmultiplexer 1101, and the low-order GATED FLIT signals, identified asGATED FLIT (1:0) signals, are coupled to a second set of input terminalsof multiplexer 1101.

As noted above, if the FLIT signals represent flits in the down pathidentification portion 41, the high-order bits in each flit, which arerepresented by the high-order GATED FLIT (3:2) signals, are used in adata router node 22(i,j,k) at an even-numbered level "i" to determinethe child to receive the data router message packet 30. On the otherhand, the low-order bits in each flit, which are represented by thelow-order GATED FLIT (1:0) signals, are used in a data router node22(i,j,k) at an odd-numbered level "i" to determine the child. Inaddition, the low-order NODE HEIGHT(0) signal; which if assertedindicates that the data router node 22(i,j,k) is at an odd-numberedlevel and if negated indicates that it is at an even-numbered level.

Accordingly, the NODE HEIGHT (0) signal controls the multiplexer 1101.If the NODE HEIGHT (0) signal is asserted, multiplexer 1101 couples theGATED FLIT (1:0) signals as SEL DN ID (1:0) selected down pathidentification signals to an input terminal of a second multiplexer1102. On the other hand, if the NODE HEIGHT (0) signal is negated, themultiplexer 1101 couples the GATED FLIT (3:2) signals as the SEL DN ID(1:0) signals.

Multiplexer 1102 receives the SEL DN ID (1:0) selected down pathidentification signals at one set of input terminals. At a second set ofinput terminals, the multiplexer 1102 receives CHILD MAP (1:0) child mapsignals. The CHILD MAP (1:0) signals are provided by the node controlcircuit 1004 (FIG. 11A) to identify, for each of the input child andparent interface circuits 1006(i) and 1010(i), one output child circuit1007(i) to which data router message packets 30 are to be coupled whilethe data router 15 is in all-fall-down mode. The multiplexer 1102 iscontrolled by an AFD DN PA SEL all-fall-down down path select signalfrom an all-fall-down latch circuit 1103.

The all-fall-down latch circuit 1104 includes a multiplexer 1104 whichreceives the AFD MODE all-fall-down mode signal at one input terminal.If the input control circuit 1041 is asserting the MSG INIT messageinitialization signal, the multiplexer 1104 couples the AFD MODEall-fall-down mode to the input terminal of a flip-flop 1105. Theflip-flop 1105 is set or clear in response to the next tick of the NODECLK signal to generate an asserted or negated AFD DN PA SEL AFD MODEall-fall-down down path select signal. As noted above, the input controlcircuit 1041 maintains the MSG INIT signal asserted for only one tick ofthe NODE CLK signal following receipt of FLIT signals representing thefirst flit of a data router message packet 30. Thereafter, the MSG INITsignal is negated, which enables the multiplexer 1104 to couple the AFDDN PA SEL signal to the data input terminal of the flip-flop 1105.

Accordingly, the flip-flop 1105 maintains the AFD DN PA SELall-fall-down down path select signal in a constant condition after theflit flow control circuit 1040 receives the first flit of the datarouter message packet 30. If the AFD MODE all-fall-down mode signal isat that point negated, indicating that the data router 15 is not inall-fall-down mode, the flip-flop 1105 is clear, thereby negating theAFD DN PA SEL signal. On the other hand, if the AFD MODE all-fall-downmode signal is asserted, indicating that the data router 15 is inall-fall-down mode, the flip-flop 1105 is set, thereby asserting the AFDDN PA SEL signal.

If the AFD DN PA SEL signal is negated, the multiplexer 1102 is enabledto couple the SEL DN ID (1:0) selected down path identification signalas the OUT REQ [C(1:0)] output request signals. In this condition, theOUT REQ [C(1:0)] signals are derived from the FLIT signals representingflits of the down path identification portion 41 of the data routermessage packet 30. If, on the other hand, the AFD MODE all-fall-downmode signal is asserted, the multiplexer 1102 is enabled to couple theCHILD MAP (1:0) signals as the OUT REQ [C(1:0)] signals.

It will be appreciated that the parent request generator portion 1090and child request generator portion 1091 will respond to FLIT signalsrepresenting all of the successive flits received by the flit flowcontrol portion 1040. The push signal generator portion 1092, whichgenerates the OIF PUSH output identification FIFO push signal, enablesthe output request identification FIFO 1023 to load the-OUT REQ[P,C(1:0)] signals when they are based on FLIT signals representing theheader field 40 and first flit of the down path identification portion41. The push signal generator portion includes a multiplexer 1110 thatreceives at one input terminal the FIRST FLIT signal from AND gate 1052(FIG. 11B-2B). As noted above, the FIRST FLIT signal is asserted whenthe FLIT signals represent the header field 40 of the data routermessage packet 30 being received. It will be appreciated that at thatpoint the HDR<header value less than node height signal received by theAND gate 1093 (FIG. 11B-2C) corresponds to the result of the comparisonbetween the node height and the value of the header field 40 asperformed by comparator 1056.

The output of multiplexer 1110 is coupled to the data input terminal ofa flip-flop 1111, which is set in response to the next tick of the NODECLK signal to assert an OIF PUSH EN output identification FIFO pushenable signal. It will be appreciated that at that point, the flip-flop1096 in the parent request generator portion 1090 also latches thesignal from the multiplexer 1095 that represents the state of the GO UPsignal. Accordingly, the OIF PUSH EN signal is asserted at the samepoint that the OUT REQ [P] signal indicates whether the data routermessage packet 30 is to be transmitted up the tree defining the datarouter 15.

Contemporaneously, if the VAL FLIT signal is asserted, the GATED FLITsignals will represent the first flit of the down path identificationportion 41 of the data router message packet 30 being receive. Thus, theOUT REQ [C(1:0)] signals will identify a down path identifier. Since theVAL FLIT and OIF PUSH EN output identification FIFO push enable signalsare asserted, an AND gate 111 is energized to assert the OIF PUSH outputidentification FIFO push signal, which is coupled to the output requestidentification FIFO 1022 (FIG. 11B). The output request identificationFIFO 1022 loads the OUT REQ [P,C(1:0)] signals at the next tick of theNODE CLK signal.

As noted above, the FIRST FLIT signal is asserted by AND gate 1052 onlywhile the FLIT signals represent the header field 1041. Accordingly, atthat point, the FIRST FLIT signal will be negated. Since the VAL FLITsignal is asserted, multiplexer 1110 couples a negated signal to thedata input terminal of flip-flop 1111, which is reset at the next tickof the NODE CLK signal to negate the OIF PUSH. EN output identificationFIFO push enable signal. Since the OIF PUSH EN signal is negated, theOIF PUSH output identification FIFO push signal will also be negated.

It will be appreciated that the flip-flop 1096 in parent requestgenerator portion 1090 and flip-flop 1111 in push signal generatorportion 1092 effectively correspond to delay lines. The respectiveflip-flops delay the respective GO UP and OIF PUSH EN outputidentification FIFO push enable signals so that they will be coupled tothe output request identification FIFO 1023 contemporaneously with thegeneration by the child request generator portion 1091 of the OUT REQ[C(1:0)] signals in response to FLIT signals representing the first flitof the down path identification portion 41 of the data router messagepacket 30 being received. It will be appreciated, however that the FLITsignals representing the first flit of the down path identificationportion 41 may be stalled. In that case, the VAL FLIT valid flit signalbe negated. The negated VAL FLIT signal enables the multiplexers 1095and 1110 to, in turn, enable respective flip-flops 1096 and 1111 tomaintain their respective condition at subsequent ticks of the NODE CLKsignal. The negated VAL FLIT signal also disables AND gate 1112, whichalso negates the OIF PUSH output identification FIFO push signal,inhibiting the output request identification FIFO 1022 from loading theOUT REQ [P,C(1:0)] signals.

FIGS. 11B-2D and 11B-2E depict detailed diagrams of, respectively, thedata flit counter 1043 and down path identification flit counter 1042.With reference to FIG. 11B-2D, the data flit counter 1042 comprises twocounters, namely, a binary counter 1114 and a ring counter 1115. Asnoted above, the value in the message length field 34 of a data routermessage packet 30 identifies the number of thirty-two bit wordscontained in the data flits 36 in the data portion 32, and eachthirty-two bit word is contained in eight successive four-bit flits. Inthat case, the ring counter 1115 decrements when each flit is receivedand counts out after receipt of the number of flits containing eachword. When the ring counter 1115 counts out, it enables the binarycounter 1114 to decrement. At that point, the ting counter isreinitialized and resumes decrementing while flits for the next word arebeing received. These operations are repeated until the binary counter1114 has counted out and the ring counter 1115 has almost counted out,AND gate 1116 is energized to assert the NR DATA END near data endsignal. When the ring counter 1115 later counts out, an AND gate 1117 isenergized to assert the DATA END signal.

More particularly, the FLIT signals are coupled to the data inputterminals of binary counter 1114. When the input control circuit 1041asserts the MSG LEN IN message, at which point the FLIT signalsrepresent the flit corresponding to the message length field 34 of thedata router message packet 30, the binary counter 1114 loads the FLITsignals. Contemporaneously, the ring counter 1115 is initialized to loada value corresponding to the number of flits in the data portion 32 of adata router message packet 30 are required to hold a thirty-two bit dataword. Since in one embodiment eight flits are required, the ring counter1115 has eight bits. It will be appreciated that, in embodiments havinga different number of flits for each data word, the ring counter 1115may have a corresponding different number of bits. To accommodate theadditional flit for the tag field 35, the low-order bit of the ringcounter 1115 is energized to load a value of "one," and the other bitsare de-energized to load values of "zero."

While the VAL FLIT valid signal is asserted, indicating that FLITsignals representative of flits of the data router message packet 30 arebeing received, the ring counter 115 is enabled to decrement. Whileenabled, since the flit flow control circuit 1040 receives successiveflits in synchronism with successive ticks of the NODE CLK signal, thering counter decrements at each tick of the NODE CLK signal.

At the first tick of the NODE CLK signal after being loaded, whichoccurs when the flit containing the tag field 35 is being received, thering counter 1115 sequences to energize its high-order bit. It will beappreciated that that high-order bit is energized contemporaneously withthe receipt by the flit flow control circuit 1040 of FLIT signalscorresponding to the first data flit 36. While the VAL FLIT signal isasserted, in synchrony with successive ticks of the NODE CLK signal, theflit flow control circuit 1040 receives successive data flits 36 of thedata router message packet 30 being received, and when it counts out,the number of data flits 36 have been received corresponding to thenumber of flits in a thirty-two bit word.

At that point, the ring counter 1115 asserts a FLIT/WORD DOflits-per-word DO signal, which energizes one input terminal of an ANDgate 1120. If the VAL FLIT signal is asserted, the other input terminalof AND gate 1120 is also energized to assert a WORD CNT DN word countdown signal, which enables the binary counter 1114 to decrement. Inresponse to the next-tick of the NODE CLK signal, the binary counter1114 decrements. The binary counter 1114 transmits RCVD WORD receivedword signals that identify, in binary-encoded form, the number ofthirty-two bit words to be received.

While the VAL FLIT signal is asserted, since the counter 1115 is a ringcounter, at the next tick of the NODE CLK signal after its low-order bitis energized, the high-order bit will be energized, and with successiveNODE CLK signals the bit that is energized will correspond to the numberof data flits 36 remaining to be received for the thirty-two bit dataword. If the VAL FLIT signal is negated, indicating that the receptionof flits has been stalled, the ring counter 1115 stops decrementing. Itresumes decrementing when the VAL FLIT signal is again asserted,indicating that reception of flits has resumed.

These operations continue until the RCVD WORD received word signals areall negated. At that point, the RCVD WORD signals have a binary-encodedvalue of zero, indicating that the flit flow control circuit 1040 isreceiving the data flits 36 representing the last thirty-two bit dataword in the data router message packet 30. The negated RCVD WORD signalsare complemented to energize an AND gate 1121, which asserts a LAST WORDsignal. The asserted LAST WORD signal, in turn, enables one inputterminal of AND gates 1116 and 1117. Contemporaneously with thereception by the flit flow control circuit of the third to last dataflit 36 in the data portion 32 of the data router message packet 30being received, the ring counter 1115 asserts a FLIT/WORD D2 flits perword D2 signal, which enables the second input terminal of AND gate1116. This energizes the AND gate to assert the NR DATA END near dataend signal. As described above, the flit flow control circuit 1040 usesthis signal in generating the NEOM near end-of-message signal.

Thereafter, contemporaneously with the reception by the flit flowcontrol circuit of the last data flit 36 in the data portion 32, thering counter asserts the FLIT/WORD DO flits per word DO signal, whichenables the second input terminal of AND gate 1117. This energizes theAND gate to assert the DATA END signal. As described above, the flitflow control circuit 1040 uses this signal in generating the EOMend-of-message signal.

It will be appreciated that the ring counter 1115 also generates theFLIT/WORD D2 and FLIT/WORD DO signals contemporaneously with receptionby the flit flow control circuit 1040 of the third to last and lastflits 36 of the data portion 32. However, except during reception of thelast thirty-two bit data word, the AND gate 1121 is de-energized, whichmaintains the LAST WORD signal negated, which consequently maintains theAND gates de-energized and the NR DATA END and DATA END signals negated.

FIG. 11B-2E depicts a detailed logic diagram of the down pathidentification flit counter 1042. With reference to. FIG. 11B-2E, thecounter 1042 includes a decoder portion 1122 and a counter portion 1123.The decoder portion generates a plurality of CNT (i) count signals ("i"is an integer from zero to eight) that identifies the number of flits inthe down path identification portion 41 of the data router messagepacket 30 being received. The counter portion 1123 decrementscontemporaneously with the receipt by the flit flow control circuit 1040of FLIT signals representing the successive flits of the down pathidentification portion, and generates the DNF END down flit end signalwhen the last flit in portion 41 is being received.

As described above, the header field 40 in a data router message packet30 contains a value that essentially identifies the number of down pathidentification fields 42 in the down path identification portion 41. Inaddition, each flit in the down path identification portion 41 includestwo down path identification fields 42. Thus, if the value in headerfield 40 is an even number, the number of flits in the down pathidentification portion is one-half the value in the header field 40. Onthe other hand, if the value in the header field 40 is an odd number,the number of flits in the down path identification portion is the oneplus the greatest integer in one-half the value in the header field 40.The decoder portion 1122 energizes the CNT (i) count signal whose index"i" corresponds to this number.

In particular, the decoder portion 1122 includes a decoder 1128 thatreceives the high-order FLIT (3:1) signals and generates in responsethereto HALF FLIT (i) signals ("i" is an integer from zero to eight).The high-order FLIT (3:1) signals represent a binary-encoded valuecorresponding to the greatest integer in one half of the binary-encodedvalue of the four-bit FLIT signals. The decoder 1128 asserts one of theHALF FLIT (i) signals whose index "i" corresponds to this value.

The decoder portion 1122 also includes a set of multiplexers 1124(i)("i" is an integer from zero to eight). Each multiplexer 1124(i)generates one of the CNT (i) count signals of corresponding index "i".In addition, each multiplexer 1124(i) receives, at one input terminalthe HALF FLIT (i) signal and at the other input terminal the HALF FLIT(i-1) signal from the decoder 1128.

The multiplexers 1124(i) are controlled in parallel by the low-orderFLIT (0) signal, which, if negated, indicates that the down pathidentification portion 41 includes an even number of down pathidentification fields 42, and if asserted indicates that it includes anodd number of down path identification fields 42. If the FLIT (0) signalis negated, indicating that the down path identification portion 41contains an even number of flits, it enables the multiplexers 1124(i) tocouple the HALF FLIT (i) signals as the CNT (i) signal, so that the oneof the CNT(i) signals that is asserted corresponds to one-half the valueof the header field 40. On the other hand, if the FLIT (0) signal isasserted, indicating that the down path identification portion 41contains an odd number of flits, it enables the multiplexers 1124(i) tocouple the HALF FLIT (i-1) signals as the CNT (i) signal. In that case,the one of the CNT (I) signals that is asserted corresponds to one plusone-half the value of the header field 40.

The counter portion 1123 includes a plurality of count stages 1125(i)("i" is an index from zero to eight) each of generates one DNF (i) downflit signal ("i" is an index from zero to eight) whose index "i"identifies the number of flits in the down path identification portion41 currently being received. Each stage 1125(i) includes a multiplexer1126(i) and a flip-flop 1127(i). Each multiplexer 1126(i) receives atone input terminal one of the CNT (i) signals of corresponding index"i." Each multiplexer 1126(i) provides a SEL CNT (i) selected countsignal that is coupled to the data output terminal of the flip-flop1127(i) of corresponding index "i." Each multiplexer 1126(i) also has aninput terminal connected to the data output terminal of the flip-flop1127(i) and another input terminal connected to the data output terminalof the flip-flop 1127(i+1). The multiplexers 1126(i) are controlled inparallel by the MSG INIT message initialization signal from the inputcontrol circuit 1141 and by a DNF CNT GO down flit count go signal froman AND gate 1130. The flip-flops 1127(i) are clocked in parallel by theNODE CLK signal.

Prior to and during receipt by the flit flow control circuit 1040 of theFLIT signals representing the header field 40 of a data router messagepacket 30, the MSG INIT message initialization signal enables themultiplexers 1126(i) to couple the CNT (i) count signals as SEL CNT (i)selected count signals to the data input terminals of flip-flops1127(i). When the flit flow control circuit 1040 is receiving FLITsignals representing the header field 40, the decoder portion 1122asserts one CNT (i') count signal whose index "i'" identifies the numberof flits in the down flit identification portion 41 Of the data routermessage packet 30. Each flip-flop 1127(i) latches the SEL CNT (i)signal, including the SEL CNT (i') signal that is asserted, at the nexttick of the NODE CLK signal. The one flip-flop 1127(i') is set to assertthe DNF (i') down flit signal, while the other flip-flops are cleared tonegate the DNF (i) down flit signals of other indices "i." At thatpoint, the input control circuit 1041 negates the MSG INIT signal.

The AND gate 1130 is controlled by the VAL FLIT valid flit signal andthe complement of the MSG INIT message initialization signal asgenerated by an inverter 1130. If the MSG INIT signal is asserted,inverter 1131 maintains the AND gate 1130 in a de-energized condition,so that the DNF CNT GO down flit count go signal will remain negated.However when the MSG INIT signal is negated, one input terminal of ANDgate 1130 is enabled and the other input terminal, which is controlledby the VAL FLIT valid flit signal, controls the energization of the ANDgate 1130 and thus the condition of the DNF CNT GO signal. Thus, whilethe VAL FLIT signal is asserted, indicating that successive flits arebeing received, the DNF CNT GO signal is asserted, and otherwise it isnegated.

While the DNF CNT GO signal is asserted, the multiplexers 1126(i) areenabled to couple the DNF (i+1) down flit signal as the SEL FLIT (i)signal. As noted above, the corresponding flip-flops 1127(i) latch theSEL FLIT (i) signal at each tick of the NODE CLK signal. Thus, while theDNF CNT GO signal is asserted, the index "i'" of the one SEL FLIT (i')signal that is asserted is decremented at successive ticks of the NODECLK signal. Accordingly, successive ticks of the NODE CLK signal, thecorresponding index "i" of DNF (i') down flit that is asserted is alsodecremented. It will be appreciated that at some point the flip-flop1127(1) will be set to assert the DNF (1) signal, which corresponds tothe DNF END down flit end signal.

If during this process the VAL FLIT signal is negated, indicating astall condition, the AND gate 1130 negates the DNF CNT GO down flitcount go signal. The negated DNF CNT GO signal enables the multiplexers1126(i) to couple the DNF (i) signals from their respective flip-flops1127(i) as the SEL CNT (i) selected count signals, instead of the DNF(i+1) signal from the flip-flop in the next higher indexed stage1125(i+1). Thus, each flip-flop 1127(i), including the one flip-flop1127(i') that is set, maintains it condition. When the VAL FLIT is againasserted, indicating termination of the stall condition, the AND gate1130 again asserts the DNF CNT GO signal, to enable the counter portion1123 to operate as described above.

iv. Switch Input Control Circuit

FIG. 11B-3 depicts a logic diagram of the switch input control circuit1024 (FIG. 11B). With reference to FIG. 11B-3, the switch input controlcircuit 1024 includes three primary sections. An output request section1140 controls the obtaining of switch control information from theoutput request identification FIFO 1023, decoding it, and providing thedecoded information to the switch 1003 (FIG. 11A). A message flitcontrol section 1141 controls retrieval of flits of data router messagepackets 30 stored in the input message FIFO 1022 and transmittal to theswitch 1003. In addition, the message flit control section 1141 receivesan generates control signals for controlling transfer of flits from theinput message FIFO 1022 to the message flit control section 1141 andfrom the section 1141 to the switch 1003. Finally, a control section1142 synchronizes the operations of both the output request section andthe message flit control section 1141.

The control section 1142 includes a control circuit 1143 that isessentially a state machine. FIG. 11B-3A comprises a state transitiondiagram depicting the conditions of input signals under which thecontrol circuit sequences from state to state. In each state transition,the control circuit 1143 changes state at a tick of the NODE CLK signal.Initially, the control circuit 1143 is in an idle state, as identifiedby the box labelled "IDLE," and it remains there as long as the outputrequest identification FIFO 1023 (FIG. 11B) is asserting the OIF EMPTYoutput identification FIFO empty signal. As noted above, if the outputrequest identification FIFO 1023 is asserting the OIF EMPTY signal, itis empty. If the output request identification FIFO 1123 becomes notempty, it negates the OIF EMPTY signal, and the control circuit 1143 atthe next tick of the NODE CLK signal sequences to the request pendingstate, as identified by the box labelled REQUEST PENDING. In that state,the control circuit 1143 asserts a REQ PENDING request pending signal.

Contemporaneously, the output request identification FIFO 1023 transmitsthe new request as ORIF OUT REQ [P,C(1:0)] buffered output requestsignals. Returning to FIG. 11B-3, the ORIF OUT REQ [P,C(1:0)] signalsare coupled to data input terminals of a latch 1150, which latches themin response to the next tick of the NODE CLK signal. In response to thelatched signals, the latch 1150 transmits a P REQ parent requestedsignal and binary-encoded C REQ (1:0) child request signals. The P REQparent requested signal is enables one input terminal of an AND gate1152.

The C REQ(1:0) child request signals are coupled to a decoder 1151,which decodes the C REQ (1:0) signals and transmits in response foursignals identified as C0 REQ through C3 REQ (generally identified as "CiREQ" child "i" requested signals). The decoder 1151 asserts the one ofthe Ci REQ signals ("i" having a value from zero to three) that has theindex "i" having the value identified by the binary encoding of the CREQ (1:0) signals.

The Ci REQ child "i" requested signals from decoder 1151 are coupled toa gated driver 1153. If the P REQ parent requested signal is notasserted, an inverter 1154 enables the gated driver 1153 to couple theCi REQ signals to input terminals of a second gated driver 1155. The REQPENDING request pending signal from the control circuit 1143 enables theAND gate 1152 to generate the OUT REQ P/SW output requested parent toswitch signal and the gated driver 1155 to generate the OUT REQ[C3:C0]/SW output request children to switch signals, which togetherform the OUT REQ [P,C3:C0] output request signals (FIG. 11B) that aretransmitted to the switch 1003. Thus, if the P REQ parent requestedsignal is asserted, the asserted REQ PENDING signal will enable AND gate1152 to assert the OUT REQ P/SW signal, and otherwise the OUT REQ P/SWsignal will be negated. Similarly, if the P REQ signal is negated andone of the Ci REQ child "i" requested signals is asserted, the assertedREQ PENDING signal will enable the assertion of the one "i-th" OUT REQ[Ci]/SW output request to switch signal.

It will be appreciated that the inverter 1154 ensures that the gateddriver 1153 will be disabled if the P REQ parent requested signal isasserted, ensuring that the request that the data router message packet30 be transmitted to a parent data router node 22(i,j,k) will takeprecedence over a request that it be transmitted to a child data routernode 22(i,j,k). This, in turn, ensures that the data router messagepacket 30 will be transmitted up the tree defining data router 15 untilit reaches a data router node 22(i,j,k) at the level "i" identified inthe data router message packet 30 as originally transmitted.

The REQ PENDING signal is also coupled to the data input terminal of aflip-flop 1144 in the control section 1142. The assertion of the REQPENDING signal enables the flip-flop 1144 to be set at the next tick ofthe NODE CLK signal, enabling it to assert a DEL REQ PENDING delayedrequest pending signal. The asserted DEL REQ PENDING signal enables oneinput terminal of an AND gate 1146 in the output request section. Theasserted DEL REQ PENDING signal also enables a second flip-flop 1048 tobe set at the next tick of the NODE CLK signal, enabling it to assert aDDEL REQ PENDING delayed (twice) request pending. The DEL REQ PENDINGand DDEL REQ PENDING signals are thus asserted one and two ticks,respectively, of the NODE CLK signal after assertion of the REQ PENDINGsignal. It will also be appreciated that the DEL REQ PENDING and DDELREQ PENDING signals will be negated one and two ticks, respectively,after negation of the REQ PENDING signal.

As will be described below in connection with FIGS. 11C-1 through 11C-6,when the switch 1003 receives one of the OUT REQ [P,C3:C0] signals thatis asserted, it performs an arbitration operation in connection withrequests from other input child and parent circuits 1006(i) and 1010(i)for a parent or the child identified by the asserted OUT REQ [P,C3:C0]signal. At some point, the request will be granted, at which point theswitch 1003 asserts the REQ GRANTED request granted signal. The controlcircuit 1143 receives the REQ GRANTED signal, and when assertedsequences to the request granted state at the next tick of the NODE CLKsignal (see FIG. 11B-3A). In that state, the control circuit 1143negates the REQ PENDING request pending signal and asserts a REQ GRrequest granted signal.

The asserted REQ GR request granted signal enables the second inputterminal of AND gate 1146. Since, as noted above, the DEL REQ PENDINGsignal remains asserted for one tick of the NODE CLK signal afternegation of the REQ PENDING signal, both input terminals of AND gate1146 will be enabled, thereby energizing the AND gate to assert the OIFPOP output identification FIFO pop signal. This enables the outputrequest identification FIFO 1023 to couple new ORIF OUT REQ [P,C(1:0)]signals to the output request section 1140, which can be latched anddecoded as described above and the resulting GATED C3:C0 REQ and P REQsignals coupled to the respective input terminals of gated driver 1155and AND gate 1152 to be available for gating as the OUT REQ [P,C3:C0]signals when the REQ PENDING signal is next asserted.

The asserted REQ GR request granted signal also enables an inputterminal of an AND gate 1160 in the message flit control section 1141.Since the DDEL REQ PENDING delayed (twice) request pending signal isasserted, AND gate 1160 is energized, which, in turn, energizes an ORgate 1161 to assert a FLY signal. The FLY signal is coupled to the datainput terminal of a flip-flop 1162, which is set in response to the nexttick of the NODE CLK signal to assert the SW FLY fly to switch signal,which is coupled to the switch 1003.

The FLY signal is also coupled to the input message FIFO 1022 as the IMFPOP input message FIFO pop signal. When the IMF POP signal is asserted,the input message FIFO 1022 is enabled to transmit FLIT/FIFO flit fromFIFO signals representing, at successive ticks of the NODE CLK signal,successive flits of the data router message packet 30.

At the point at which the OR gate 1161 first asserts the FLY signal, theinput message FIFO 1022 (FIG. 11B) is transmitting FLIT/FIFO flit fromFIFO signals representing the first flit of the data router messagepacket 30. The FLIT/FIFO signals are buffered in a latch 1163, and arelatched thereby in response to the ticks of the NODE CLK signal. Theoutput signals transmitted by the latch 1163 comprise the SW FLIT flitto switch signals that are transmitted to the switch 1003. While FLYsignal is negated, the input message FIFO 1022 maintains the FLIT/FIFOsignals unchanged, representing the first flit of the data routermessage packet 30, so that the SW FLIT signals continually represent thefirst flit at successive ticks of the NODE CLK signal. However, whilethe FLY signal is asserted, the asserted IMF POP signal enables theinput message FIFO 1022 to transmit the successive flits to the latch1163, which latches them and transmits them as the SW FLIT signals toswitch 1003.

As noted above, the DDEL REQ PENDING delayed (twice) request pendingsignal remains asserted for two ticks of the NODE CLK signal after thecontrol circuit 1143 negates the REQ PENDING signal. When the DDEL REQPENDING signal is negated, the AND gate 1160 is de-energized. At thatpoint, the latch 1163 will have transmitted SW FLIT flit to switchsignals representing the first two flits of the data router messagepacket 30. To enable transmission of SW FLIT signals representingadditional flits, the switch 1003 asserts the SW FLOW flow from switchsignal. The coincidence of the asserted REQ GR request granted and SWFLOW flow from switch signals energize an AND gate 1164, which maintainsthe OR gate 1161 energized.

The switch 1003 may, after receiving SW FLIT flit to switch signalsrepresenting the first two flits of the data router message packet 30,stop the flow of successive flits thereto by negating the SW FLOWsignal. If it negates the SW FLOW signal, the AND gate 1164 isde-energized, which de-energizes the OR gate 1161 in turn negating theFLY signal. At the next tick of the NODE CLK signal, the negated FLYsignal clears flip-flop 1162, which negates the SW FLY signal. Negationof the FLY signal also causes the IMF POP input message FIFO pop signalto be negated, which stops sequencing of the input message FIFO 1022.The switch 1003 may thereafter enable the flow of the successive flitsto resume by re-asserting the SW FLOW switch flow signal, whichre-energizes the AND gate 1164 and OR gate 1161 to assert the FLY andIMF POP signals, and enables the flip-flop to again be set to assert theSW FLY signal.

At some point in transmission of FLIT/FIFO flit from FIFO signalsrepresenting the successive flits of a data router message packet 30,the input message FIFO will assert the NEOM/FIFO near end-of-messagefrom FIFO signal. As indicated above, the NEOM/FIFO signal is assertedcontemporaneously with the FLIT/FIFO flit from FIFO signals representingthe third-from-last flit of the data router message packet 30. TheNEOM/FIFO signal enables one input terminal of an AND gate 1165. Sincethe FLY signal is asserted, the AND gate 1165 is energized, whichenables the direct-set input terminals of two flip-flops 1166 and 1167,in turn setting both flip-flops. The set flip-flop 1166 asserts aPRERELEASE signal.

Returning to FIG. 11B-3A, the assertion of the PRERELEASE signal enablesthe control circuit 1143 to sequence to another state. If the OFF EMPTYsignal is contemporaneously asserted, indicating that the output requestidentification FIFO 1023 is empty, the control circuit 1143 sequences tothe idle state at the next tick of the NODE CLK signal, and negates theREQ GR request granted signal. On the other hand, if the OFF EMPTYsignal is negated, the control circuit returns to the request pendingstate. In the request pending state, the control circuit 1143 alsonegates the REQ GR signal and also asserts the REQ PENDING requestpending signal.

Since in either case the REQ GR signal is negated, the AND gate 1164 isde-energized. However, the set flip-flop 1167 asserts an ALMOST DONEsignal, which maintains the OR gate 1161 in an energized condition, inturn maintaining the FLY signal asserted. At this point, since theswitch 1003 cannot, through the SW FLOW flow from switch signal, controlthe FLY and SW FLY signals, it accepts the SW FLIT signals representingthe last few flits of the data router message packet 30.

As noted above, the flip-flop 1166 asserts the PRERELEASE signal at thesame point that the SW FLIT signals from latch 1163 represent thethird-from-last flit of the data router message packet 30. The assertedPRERELEASE signal also enables the data input terminal of a flip-flop1168, which is set in response to the next tick of the NODE CLK signal.The set flip-flop 1168 asserts the SW RELEASE release to switch signal.It will be appreciated that the SW RELEASE signal is assertedcontemporaneous with the second-to-last flit of the data router messagepacket 30. The SW RELEASE signal also controls the direct-reset inputterminal of flip-flop 1167, and resets the flip-flop when the signal isasserted. The reset flip-flop 1167 negates the PRERELEASE signal, whichenables the flip-flop 1168 to be reset in response to the next tick ofthe NODE CLK signal. It will be appreciated that at that point, the SWFLIT flit to switch signals will represent the last flit of the datarouter message packet 30.

At the same time, the input message FIFO 1022 will be asserting theEOM/FIFO end-of-message from FIFO signal. Since the ALMOST DONE signalis still asserted, energizing OR gate 1163, the FLY signal is alsoasserted. The coincidence of assertion of the EOM/FIFO and FLY signalsenergizes an AND gate 1169, which energizes the direct-reset inputterminal of flip-flop 1167. This enables the flip-flop 1167 to be reset,in turn negating the FLY and IMF POP input message FIFO pop signals. Atthe next tick of the NODE CLK signal, the flip-flop 1163 is reset,negating the SW FLY signal.

As noted above, the control circuit 1143 sequences from the requestgranted state to either the idle state or the request pending state atthe same point that the SW FLIT signals represent the third-to-last flitof the data router message packet 30, the particular state depending onthe state of the OIF EMPTY signal. If the control circuit 1143 sequencesto the idle state, the switch input control circuit 1024 can repeatoperations, as described above, when the OIF EMPTY signal is negated.If, on the other hand, the control circuit 1143 sequences to the requestpending state, it will be appreciated that the REQ PENDING signal willbe contemporaneously asserted, enabling the AND gate 1152 and gateddriver 1155 to couple OUT REQ [P,C3:C0]/SW output request to switchsignals to the switch 1003. As will be described below in connectionwith FIGS. 11C-1 through 11C-6, the switch 1003 may assert the REQGRANTED signal, enabling the control circuit 1143 to sequence to therequest granted state, immediately after the message flit controlcircuit 1141 transmits SW FLIT flit to switch signals representing thelast flit of a data router message packet 30, to enable the message flitcontrol circuit 1141 to immediately begin transmitting SW FLIT signalsrepresenting the first flit of the next data router message packet 30.

As described above, the input parent circuits 1010(i) are similar to theinput child circuits 1006(i) described above in connection with FIGS.11B through 11B-3A, with one exception noted here. In particular, asdescribed above, when data router nodes 22(i,j,k) begin passing a datarouter message packet 30 down the tree defining data router 15, thenodes 22(i,j,k) do not thereafter pass the packet 30 back up the tree.If an input parent circuit 1010(i) receives a data router message packet30 the packet 30 is being passed down the tree. Thus, the input parentcircuit 1010(i) will not enable the switch 1003 to couple the packet 30to an output parent circuit 1011(i), since that would pass the packet 30back up the tree. Accordingly, the input parent circuits 1010(i) neednot include circuitry, in their respective input message controlcircuits 1021 and switch input control circuits 1024 for generatingsignals corresponding to the OUT REQ P output request parent signals inthe OUT REQ [P,C(1:0) signals and the OUT REQ [P,C3:C0]/SW signals asdescribed above in connection with FIGS. 11B through 11B-3A.

3. Switch 1003 i. General

FIGS. 11C-1 and 11C-2 together depict, in general block diagram form,switch 1003 (FIG. 11A) in a data router node 22(i,j,k). The switch 1003includes a control section 1200 shown in FIG. 11C-1 and a switchingsection 1201 shown in FIG. 11C-2. With reference initially to FIG.11C-2, the switching section 1201 includes a plurality of switch cells,each generally identified herein by reference numeral 1202(x,y). Thedetails of a switch cell 1202(x,y) will be described below in connectionwith FIG. 11C-6. As depicted in FIG. 11C-2, the switch cells arearranged in a matrix having a plurality of rows and columns. Theswitching cells in each row are associated with a particular input childor parent circuit 1006(i) or 1010(i) (FIG. 11A) identified by themnemonic identifier "ICi" (input child "Ci") or "IPi" (input parent"Pi"), where "i" is an index having values from zero to three. Theswitching cells in each column depicted in FIG. 11C-2 are associatedwith a particular output child or parent circuit 1007(i) or 1011(i),which are identified by the mnemonic identifier "OCi" (output child"Ci") or IPi (output parent "Pi"), where "i" is an index having valuesfrom zero to three. In the reference numeral 1202(x,y) for a switchingcell, the index "x" refers to the input child or parent circuitassociated with the cell's row, and the index "y" refers to the outputchild or parent circuit associated with the row's column.

Each switch cell 1202(x,y) selectively couples the SW FLIT flit toswitch signals from the input child or parent circuits 1006(i) or1010(i) that is associated with the cell's row to the output child orparent circuit 1007(i) or 1011(i) that is associated with the cell'scolumn, under control of enabling signals from the control section 1200.The switch cell 1202(x,y) couples the SW FLIT signals received therebyas SW FLIT/Ci switched flit to child "Ci" signals or SW FLIT/Pi switchedflit to parent "Pi" signals, depending on the output child or parentcircuit 1007(i) or 1011(i) connected thereto. For example, switch cell1202(C0,C3) selectively couples the SW FLIT signals from input childcircuit 1006(0) (which are identified on FIG. 11C-2 as C0/SW FLITsignals, where the "C0" ahead of the slash identifies the source of theSW FLIT signals) as SW FLIT/C3 switched flit to child C3 signals tooutput child circuit 1007(3). Similarly, switch cell 1202(C0,P0)selectively couples the SW FLIT signals from input child circuit 1006(0)to as SW FLIT/output parent circuit

Each switch cell 1202(x,y) also receives the SW FLY fly to switch signalfrom the corresponding input child or parent circuits 1006(i) and1010(i) and selectively couples it to the output parent or child circuit1007(i) or 1011(i) connected thereto as the SW FLY/Ci switched fly tochild "Ci" signal or SW FLY/Pi switched fly to parent "Pi" signal. Theswitch cell 1202(x,y) also receives a SW FLOW/pi flow to switch fromparent "Pi" signal or SW FLOW/Ci flow to switch from child "Ci" signal,as appropriate, from the input child or parent circuit 1011(i) or1007(i) and selectively couples it to the input child or parent circuit1006(i) or 1010(i) connected thereto as the SW FLOW/Ci flow from switchto child "Ci" signal or SW FLOW/Pi flow from switch to parent "Pi"signal. With reference to FIG. 11B, the input child circuit 1006(i) ofchild "Ci" interface circuit 1001(i) (FIG. 11A), for example, receivesthe SW FLOW/Ci signal as the SW FLOW flow from switch signal. Each inputparent circuit 1010(i) receives the SW FLOW/Pi signal similarly.

Each switch cell 1202(x,y) also receives a Ci/SW RELEASE switch releasefrom child "Ci" signal or the Pi/SW RELEASE switch release from parent"Pi" signal from the input child or parent circuit 1006(i) or 1010(i)connected thereto. Each Ci/SW RELEASE signal is directed to all of theswitch cells 1202(Ci,y) and, when asserted disables the one that iscoupling the Ci/SW FLIT switch to flit from child "Ci" signals to anoutput child or parent circuit 1007(i) or 1011(i) connected thereto. Atthat point, the switching section 1200 generates a "Y" SEL EN selectenable signal indicating that the cells 1202(x,y) connected to outputchild or parent circuit 1007(y) or 1011(y) are available for selection.

It should be noted that the columns associated with the output parentcircuits 1011(i) in the embodiment depicted in FIG. 11C-2 do not includeincludes switching cells in rows associated with the input parentcircuits 1010(i). In that embodiment, as noted above, when the datarouter nodes 22(i,j,k) begin directing a data router message packet 30down the tree defining data router 15, they do not thereafter direct thepacket 30 back up the tree. Thus, if the switching circuit 1201 receivesa data router message packet 30 from an input parent circuit 1010(i), itwill not direct the packet 30 to an output parent circuit 1011(i), andso the switching circuit 1201 does not need switching cells 1202(x,y)therefor.

The control section 1200, which is shown generally in FIG. 11C-1,generates selection control signals that selectively enable the cells1202(x,y). In this operation, the control section 1200 uses the Ci OUTREQ [P,C3:C0]/SW output request from child "Ci" to switch signals and PiOUT REQ [P,C3:C0]/SW output request from parent "Pi" to switch signalsfrom the input child and parent circuits 1006(i) and 1010(i), and P3:P0SEL EN output parent selection enable signals and Ci SEL EN output childselection enable signals from the switching section 1201. {The "Ci" or"Pi" prefix in the mnemonic signal identifiers Ci OUT REQ [P,C3:C0]/SWand Pi OUT REQ [P,C3:C0]/SW identifies the source input child or parentcircuit 1006(i) or 1010(i). Thus, for example, the Ci OUT REQ[P,C3:C0]/SW signals, for Ci corresponding to C0, comprise the OUT REQ[P,C3:C0]/SW depicted on FIG. 11B.}. In response to all of thesesignals, the control section 1200 generates P3:P0 SEL [C3:C0] selectionsignals and OCy SEL [P3:P0,C3:C0] selection signals ["y" is an indexidentifying the particular output child circuits 1006(y)].

The switch control section 1200 includes five circuits, including oneparent arbitration circuit 1210 and four child arbitration circuits1211(y) ("y" being an index having integer values from zero to three).The parent arbitration circuit 1210 receives C3:C0 REQ P child requestsparent signals and P3:P0 SEL EN parent select enable signals andgenerates the P3:P0 SEL [C3:C0] output parent selection signals inresponse. The C3:C0 REQ P signals comprise the parent request portionsof the Ci OUT REQ [P,C3:C0]/SW output request from child "Ci" to switchsignals from the input child circuits 1006(i).

The P3:P0 SF-L [C3:C0] selection signals generated by the parentarbitration circuit 1210 comprise sixteen signals that control theswitching cells in the columns associated with the output parentcircuits 1011(i). Each signal, which has a mnemonic identifier of theform "Py SEL Cx," when asserted enables the switching cell 1202(x,y). Itwill be appreciated that one Py SEL Cx signal may be asserted at anygiven time for each value of "y". This ensures that SW FLIT signals fromonly one input child circuit 1006(i) are coupled to an output parentcircuit 1011(i) at any given time.

The parent arbitration circuit 1210 also transmits the P3:P0 SEL [C3:C0]selection signals to the input child circuits 1006(i) as P GRANTS[C3:C0] parent grants child signals. The P GRANTS [C3:C0] signalscomprise four signals, each identified by the mnemonic "P GRANTS Ci,"one associated with each input child circuit 1006(i). When the parentarbitration circuit 1210 asserts a P3:P0 SEL [C3:C0] selection signal toenable a switching cell 1202(x,y) in the row of associated with theinput child circuit 1006(i), it also asserts the P GRANTS Ci signal.

Each child arbitration circuit 1211(y) is associated with one column ofswitching cells 1202(x,y), identified by index "y." Each childarbitration circuit 1211(y) receives P3:P0,C3:C0 REQ Cy parent and childrequest child signals and the OCy SEL EN output select enable signalfrom the associated column of switching cells 1202(x,y). In response,the child arbitration circuit 1211(y) generates the OCy SEL[P3:P0,C3:C0] output child selection signals for that column.

The OCy SEL [P3:P0,C3:C0] signals actually comprises eight signals,having the general mnemonic identifiers "OCy SEL Px" and "OCy SEL Cx",where "x" identifies a input child or parent circuit and, thus, aparticular row of switching cells 1202(x,y) in the switching section1201. Each signal OCy SEL Px and OCy SEL Cx, when asserted, enables theswitching cell 1202(x,y). It will be appreciated that only one of theOCy SEL Px and OCy SEL Cx signals can be asserted at any given time.This ensures that SW FLIT signals from only one input child or parentcircuit 1006(i) and 1010(i) are coupled to an output child circuit1007(i) at any given time.

Each child arbitration circuit 1211(y) also transmits the OCy SEL[P3:P0,C3:C0] selection signals to the input child and parent circuits1006(i) and 1010(i) as Cy GRANTS [P3:P0,C3:C0] child grants parent andchild signals. The Cy GRANTS [P3:P0,C3:C0] signals comprise eightsignals, each identified by the mnemonic "Cy GRANTS Pi," one associatedwith each input parent circuit 1010(i), or "Cy GRANTS Ci", oneassociated with each input child circuit 1006(i). When the childarbitration circuit 1211(y) asserts an OCy SEL [P3:P0,C3:C0] selectionsignal to enable a switching cell 1202(x,y) in the row associated withthe input child or parent circuit 1006(i) or 1010(i), it also assertsthe Cy GRANTS Pi or Cy GRANTS Ci signal.

The switch control section 1200 also ORs together the P GRANTS Ci and CyGRANTS Ci signals associated with each input child circuit 1006(i) toform the Ci REQ GRANTED signal, which is coupled to the input childcircuit 1006(i) as the REQ GRANTED signal (FIGS. 11B and 11B-3). The ORoperation for input child circuit 1006(0) is represented in FIG. 11C-1by an OR gate 1212(0). It will be appreciated that the switch controlsection 1200 will include an OR gate to generate the corresponding CiREQ GRANTED signal for each of the other input child circuits 1006(i) inresponse to the P GRANTS Ci and Cy GRANTS Ci signals. In addition, theswitch control section with include an OR gate to generate thecorresponding Pi REQ GRANTED signals for each of the input parentcircuits 1010(i) in response to the Cy GRANTS Pi signals.

The parent arbitration circuit 1210, which will be described in moredetail below, essentially comprises a two-dimensional priority chain.The circuit * * *

Each child arbitration circuit 1211(i), which will be described in moredetail below in connection with FIGS. 11C-3 and 11C-4, essentiallycomprises a one-dimensional priority chain. In this connection, eachchild arbitration circuit 1211(y) assigns up to eight requests, eachrepresented by one of the eight P3:P0,C3:C0REQ Ci signals, to oneresource, which is represented by the column of switching cells1202(x,y) associated with the output child or parent circuit 1007(y) or1011(y). The child arbitration circuits 1211(y) grants access to theresource on a priority basis, but where the priority rotates among therequesters so that no requester can be inhibited from accessing theresource for an undue length of time by large numbers of requests fromother requesters.

ii. Switch Control Section

a. Child Arbitration Circuit 1211(i)

The details of a child arbitration circuit 1211(i) will be described inconnection with FIGS. 11C-3 and 11C-4. With reference to FIG. 11C-3, thechild arbitration circuit 1211(i) controlling access by the input childand parent interface circuits includes a binary arbitration tree 1213,an output circuit 1214 and a round-robin counter circuit 1215. Theround-robin counter circuit 1215 generates binary-encoded D(2:0) signalsrepresenting values from zero to seven, each associated with one of theeight request signals, that is, the four Cx REQ Ci input child "Cx"requests output child "Ci" signals and four Px REQ Ci input parent "Px"requests output child "Ci" signals, that are received by the childarbitration circuit 1211(i).

The arbitration tree 1213 receives the eight Cx REQ Ci and Px REQ Cirequest signals and selects one identified by the D(2:0) signals fromthe round-robin counter circuit 1215, and asserts a Cx SEL input child"Cx" select signal or a Px SEL input parent "Px" select signal inresponse. The output circuit 1214 asserts a corresponding Ci SEL Cxoutput child "Ci" selects input child "Cx" signal or Ci SEL Px outputchild "Cx" selects input parent "Px" signal when the switching section1201 asserts the Ci SEL EN output child "Ci" select enable signal.Contemporaneously, the output circuit 1214 asserts the corresponding CiGRANTS Cx or Ci GRANTS ex signal, which, as described above inconnection with FIG. 11C-1, is used by the switch control section 1200in generating the Cx REQ GRANTED signal for transmission to the selectedinput child circuit 1006(x), or a corresponding signal for transmissionto the selected input parent circuit 1010(x). As described above inconnection with FIGS. 11B through 11B-3A, the input child or parentcircuit 1006(x) or 1010(x) then negates the Cx REQ Ci or Px REQ Cirequest signal.

If the selected input parent or child circuit 1006(i) or 1010(i) is notasserting the one of the. Cx REQ Ci or Px REQ Ci signals associated withthe current value of the D(2:0) signals, the round-robin counterincrements until the value of the D(2:0) signals is associated with anasserted Cx REQ Ci or Px REQ Ci signal. When the counter 1215 reachessuch a value, it stops incrementing. In addition, the arbitration tree1213 at that point asserts the Cx SEL input child select signal or PxSEL input parent select signal that is associated with thatbinary-encoded value of the D(2:0) signals. Thus, the round-robincounter 1215 ensures that priority among the Cx REQ Ci and Px REQ Cisignals rotates, so that the input child and parent circuits 1006(i) and1010(i) all have a reasonably equal likelihood of being selected, andthat none are inhibited from coupling data router message packets 30 foran unduly long time.

Generally, the arbitration tree 1213 comprises arbitration cellsgenerally identified by reference numeral 1216(i,j) organized aplurality of levels where index "i" identifies the level of thearbitration cell in the arbitration tree 1213 and index "j" uniquelyidentifies the arbitration cell among others in the same level. Eacharbitration cell 1216(i,j) performs two general operations. First, eacharbitration cell 1216(i,j) receives request signals from two input childor parent circuits 1006(i) or 1010(i), or from two arbitration cells1216(i-1) in the next lower level, and generates a consolidated requestsignal that is the OR thereof.

Second, each arbitration cell 1216(i,j) performs a pair-wise arbitrationdetermination in response to (a) the request or consolidated requestsignals from the input child or parent circuits 1006(i) and 1010(i), orthe consolidated request signals, (b) unary preference signals thatgenerated by the next lower level 1216(i-1,j), and (c) the D(i) signalsfor the particular level (i). In that operation, the arbitration cell1216(i,j) generates unary preference signals to identify one of theinput child or parent circuits 1006(x) or 1010(x) that is asserting arequest signal, for use by the arbitration cell 1216(i+1,j) in thearbitration tree 1213. Thus, each arbitration cell 1216(i,j) produces apreference signal for each of the input child or parent circuits 1006(i)or 1010(i) that is connected to those arbitration cells 1216(0,j) in thefirst level which are in the sub-tree depending from the arbitrationcell 1216(i,j).

For example, the arbitration cell 1216(0,0) in the first level receivesC0 REQ Ci and C1 REQ Ci input child "C0" and "C1" request output child"Ci" signals and asserts a C0/C1 REQ Ci input child "C0" or "C1"requests output child "Ci" signal if either of the C0 REQ Ci or C1 REQCi signals are asserted. In addition, arbitration cell 1216(0,0) assertsthe one of unary-encoded C0/C1 PREF input child "C0" or "C1" preferredsignals to identify child "C0" or child "C1" as being preferred. TheC0/C1 PREF signals actually comprises two signals, one associated withthe input child circuit "C0" 1006(0) and the other associated with inputchild "C1" 1006(1), with at most one signal being asserted. Thearbitration cell 1216(0,0) selects at most one of the C0/C1 PREF signalsto be asserted in response to the conditions of the C0 REQ Ci and C1 REQCi request signals, C0 PREF and C1 PREF child "C0" or "C1" preferredsignals and a low-order D(0) round-robin count signal from round-robincounter circuit 1215.

The other arbitration cells in the first level, namely cells 1216(1)through 1216(3) operate similarly. It will be appreciated that D(0)round-robin select signal enables the arbitration cells 1216(0,j) in thefirst level, if both Cx REQ Ci or Px REQ Ci signals received thereby areasserted, to select one of the corresponding input child or parentcircuit whose preference signal is to be asserted. If the D(0) signal isasserted, the arbitration cell 1216(0,j) will assert the preferencesignal that is associated with the input child circuit 1006(x) whoseindex "x" is odd. On the other hand, if the D(0) signal is negated, thearbitration cell 1216(0,j) will assert the preference signal that isassociated with the input child circuit 1006(x) whose index "x" is zeroor even.

The arbitration cell 1216(1,0) in the second level receives (a) theC0/C1 REQ Ci input child "C0" or "C1" requests output child "Ci" signalfrom arbitration cell 1216(0,0) and (b) the C2/C3 REQ Ci input child"C0" or "C1" requests output child "Ci" signal from arbitration cell1216(0,1) and generates in response a C3:C0 REQ Ci input child requestsoutput child "Ci" signal. The C3:C0 REQ Ci signal is asserted if any ofthe Cx REQ Ci signals is asserted.

In addition, the arbitration cell 1216(1,0) generates C3:C0 PREFpreference signals, which comprises four unary-encoded signals eachassociated with one of the input child circuits 1006(3) through 1006(0).If the arbitration cell is asserting the C3:C0 REQ Ci signal, it alsoasserts one of the C3:C0 PREF signals. The arbitration cell 1216(1,0)uses the C0/C1 PREF and C2/C3 PREF preference signals which it receivesfrom the arbitration cells 1216(0,0) and 1216(0,1) in its sub-tree,along with the D(1) signal from the round-robin counter 1215. As notedabove, at most one of the C0/C1 PREF signals asserted, and similarly atmost one of the C2/C3 PREF signals will be asserted. If one of thesignals in each pair of C0/C1 PREF signals and C2/C3 PREF signals isasserted, the D1 signal is used to select one of the C3:C0 PREF signalsto be asserted. The arbitration cell 1216(1,1) operates similarly togenerate the P3:P0 REQ input parent request signal and the P3:P0 PREFinput parent preferred signals.

Finally, the arbitration cell 1216(2,0) at the root of the arbitrationtree 1213 operates similarly to generate an C3:C0/P3:P0 REQ Ci inputchild/input parent requests output child "Ci" signal in response to theC3:C0 REQ Ci input child requests output child "Ci" signal and P3:P0 REQCi input parent requests output child "Ci" signal. In addition, thearbitration cell 1216(2,0) operates similarly, in response to the C3:C0REQ Ci and the P3:P0 REQ Ci request signals, the C3:C0 PREF and theP3:P0 PREF preference signals, and the D(2) signals, to generateunary-encoded Cx SEL input child "Cx" select signals and Px SEL inputparent "Px" select signals. If one of the C3:C0 PREF preference signalsand one of the P3:P0 PREF preference signals is asserted, the D(2)signal will determine which of the Cx SEL signals or Px SEL signals willbe asserted. If the D(2) signal is asserted, the arbitration cell willassert one of the Px SEL signals, corresponding to the one of the P3:P0PREF preference signals that is asserted. In addition, if the D(2)signal is negated, the arbitration eel] 1216(2) will assert one of theCx SEL signals, corresponding to the one of the C3:C0 PREF preferencesignals that is asserted.

As noted above, the round-robin counter circuit 1215 generatesbinary-encoded D(2:0) signals having values between zero and seven, eachof which is associated with one of the Cx REQ Ci or Px REQ Ci requestsignals from the input child and parent circuits 1006(i) or 1010(i). Ifthe Cx REQ Ci or Px REQ Ci request signal associated with the currentbinary-encoded value of the D(2:0) signals is not asserted, theround-robin counter increments until the value of the D(2:0) signals isassociated with an asserted Cx REQ Ci or Px REQ Ci signal. As shown inFIG. 11C-3, the round-robin counter circuit 1215 includes a binarycounter 1217, a multiplexer 1220 and an inverter 1221. The binarycounter 1217 generates the D(2:0) signals, which are identified in FIG.11C-3 as signals D(2), D(1) and D(0).

In addition to being directed to the arbitration tree 1213, the D(2:0)signals are also directed to control input terminals of multiplexer1220. The data input terminals of multiplexer 1220 receive the Cx REQ Ciand Px REQ Ci request signals, and the multiplexer 1220 couples as a SELREQ selected request signal the one associated with the binary-encodedvalue of the D(2:0) signals. The inverter 1221, which controls anincrement enable terminal of the counter 1217, complements the SEL REQselected request signal, to enable the counter 1217 if the SEL REQ, andthus the selected Cx REQ Ci or Px REQ Ci signal, is negated. If thecounter 1217 is enabled, it increments in response to successive ticksof the NODE CLK signal. Thus, the SEL REQ signal is asserted, enablinginverter 1221 to disable counter 1217, when the D(2:0) signals identifyone of the Cx REQ Ci or Px REQ Ci signals that is asserted.

The output circuit 1214 includes a plurality of AND gates 1222(0)through 1222(7), each of which generates a Ci SEL Cx or Ci SEL Px signaland an associated Ci GRANTS Cx or Ci GRANTS Px signal. Each AND gate1222(i) asserts its respective output signals in response to thecoincidence of the corresponding Cx SEL or Px SEL signal and theC3:C0,P3:P0 REQ CI signal from the arbitration cell 1216(2,0) and the CiSEL EN select enable signal from the switching section 1201 (FIG.11C-2). In addition, the output circuit 1214 includes an AND gate 1223which generates a Ci TAKEN signal in response to the coincidence of theC3:C0,P3:P0 REQ CI signal and the Ci SEL EN select enable signal. The CiTAKEN signal is coupled to the node control circuit 1004 (FIG. 11A) toindicate when the arbitration circuit 1211(i) for a particular outputchild circuit 1007(i) has selected one of the input child or parentcircuits 1006(i) or 1010(i).

FIG. 11C-4 depicts a logic diagram of an arbitration cell 1216(0,0) inthe first level of the arbitration tree 1213. The other arbitrationcells 1216(i,j) are similar. With reference to FIG. 11C-4, thearbitration cell includes an OR gate 1224 that receives the C0 REQ Ciand C1 REQ Ci request signals from the input child circuits 1006(0) and1006(1) and generates the C0/C1 REQ Ci consolidated request signal ifeither input signal is asserted.

The C0 REQ Ci and C1 REQ Ci signals are also coupled, along with theD(0) signal from round-robin counter 1217, to a selector circuit 1225.The selector circuit includes an AND gate 1226 that generates a SEL C0PREF EN select child "C0" preferred enable in response to thecoincidence of the C0 REQ Ci request signal and the complement of a SELC1 select child "C1" signal from an inverter 1226. The SEL C1 signal, inturn, is generated by an AND gate 1230. If either or both of the DO orC1 REQ Ci signals is negated, the AND gate 1230 is disabled to negatethe SEL C1 signal. The negated SEL C1 signal is, in turn, complementedby the inverter 1227 to enable one input terminal of AND gate 1226.Thus, if the C0 REQ Ci signal is asserted, the AND gate 1226 asserts theSEL C0 PREF EN signal.

The asserted SEL C0 PREF EN signal, in turn, energizes one inputterminal of an AND gate 1231, whose other input terminal is maintainedin an enabled condition by an asserted C0 PREF child "C0" preferredsignal. The AND gate 1231 is thus energized to assert the one of theC0/C1 PREF preferred signals associated with input child circuit1006(0). The asserted SEL C0 PREF EN signal, in turn, is complemented byan inverter 1232 to negate a SEL C1 PREF EN select child "C1" preferredenable signal. The negated SEL C1 PREF EN signal disables an AND gate1233, causing it to negate the one of the C0/C1 PREF preferred signalsassociated with input child circuit 1006(1).

On the other hand, if both of the C1 REQ Ci and the D(0) signals areasserted, the AND gate 1230 is energized to assert the SEL C1 signal.The asserted SEL C1 signal, in turn, enables the inverter 1227 todisable AND gate 1226, regardless of the condition of the C0 REQ Cisignal, to maintain the SEL C0 PREF EN signal in a negated condition.The negated SEL C0 PREF EN signal disables AND gate 1231, causing it tonegate the one of the C0/C1 PREF preferred signals associated with inputchild circuit 1006(0). On the other hand, inverter 1232 asserts the SELC1 PREF EN signal, which enables one input terminal of AND gate 1233.The other input terminal of AND gate 1233 is maintained in an enabledcondition by an asserted C1 PREF child "C1" preferred signal. The ANDgate 1233 is thus energized to assert the one of the C0/C1 PREF signalsassociated with the input child circuit 1006(1).

As noted above, the other arbitration cells 1216(i,j) are similar. Inarbitration cells 1216(i,j) in the second and third levels (for which"i" in the respective reference numeral is "1" or "2"), it will beappreciated that there will be multiple AND gates for each of AND gates1231 and 1233, one for each of the unary preferred signals received bythe respective cell, controlled in parallel by the signals correspondingto the SEL C0 PREF EN and SEL C1 PREF EN signals.

b. Parent Arbitration Circuit 1210

FIG. 11C-5 depicts the detailed circuit of the parent arbitrationcircuit 1210. With reference to FIG. 11C-5, the parent arbitrationcircuit 1210 includes a child request priority circuit 1240, a parentavailability priority circuit 1241 and a child request/parentavailability match circuit 1242. Generally, the child request prioritycircuit 1240 receives the C3:C0 REQ P child "Ci" requests parent signalsfrom the input child circuits 1006(i) and establishes prioritiesthereamong. The priorities periodically change, on a round-robin basis.The parent availability priority circuit 1241 receives the P3:P0 SEL ENparent select enable signals from the switching section 1201 andestablishes priorities thereamong. The priorities established by theparent availability priority circuit also change periodically, but thepriorities are established on a generally random basis. Finally, thechild request/parent availability match circuit matches child requestswith parents available in the respective priorities, and generates theP3:P0 SEL [C3:C0] signals in response.

More specifically, the child request priority circuit 1240 receives theC3:C0 REQ P child "Ci" request parent signals from the input childcircuits 1006(i) and generates, for each, a set of Ci REQ P PRI (3:0)child "Ci" requests parent priority signals. The Ci REQ P PRI (3:0)signals for each input child circuit 1006(i) comprise a plurality ofpriority signals identified by the mnemonic Ci REQ P PRI (x), eachrepresentative of a priority level. The Ci REQ P PRI (x) signals foreach input child port 1006(i), and for descending values of "x"represents descending priority levels for the input child port 1006(i).

The child request priority circuit 1240, in response to assertion by atleast one input child circuit 1006(i) of a Ci REQ P signal, determines apriority ranking among the input child circuits. In that operation, thechild request priority circuit 1040 makes use of a round-robin numbergenerator 1243 and a child request enumerator 1244. The round-robinnumber generator 1243 generates RND-RBN PRI round-robin priority signalsrepresenting a value "i" which identifies one of the input childcircuits 1006(i') as having the highest priority. The child requestenumerator 1244 receives the RND-RBN PRI signals and asserts the Ci' REQP PRI (3) signal for that input child circuit 1006(i'). The childrequest enumerator 1244 also negates the Ci REQ P PRI (3) signals forthe other input child circuits 1006(i) (where "i" does not equal "i").

If any of the other input child Circuits 1006(i) are asserting their CiREQ P child "Ci" requests parent signals, the child request enumerator1244 asserts the Ci REQ P PRI (x) signals, for "x" and "i" both indescending order (with the value of "i" returning to and descending fromthe maximum value if the value represented by the value of the RND-RBNPRI signal is less than the maximum value of "i"). Thus, if, forexample, the input child circuits 1006(0), 1006(2) and 1006(3) areasserting their Ci REQ P signals, and if the RND-RBN PRI signal has thevalue "i" equalling "two," indicating that input child circuit 1006(2)has the highest priority, the child request enumerator 1244 will assertthe C2 REQ P PRI (3) signal, representative of the highest priority. Thechild request enumerator will negate the Ci REQ P PRI (3) signals forthe other input child circuits.

Continuing with the example, since the input child circuit 1006(1) isnot asserting its C1 REQ P signal, the child request enumerator 1244will negate all of the C1 REQ P PRI (3:0) signals. The assertion by theinput child circuit 1006(1) of the C1 REQ P signal will enable the childrequest enumerator 1244 to assert the C0 REQ P PRI (2) signal, and tonegate the Ci REQ P PRI (2) signals for other input child circuits.Further, since the input child circuit 1006(3) is the lowest priority asidentified by the value represented by the RND-RBN PRI round-robinpriority signal, the asserted C3 REQ P signal enables the child requestenumerator to assert the C3 REQ P PRI (1) signal, and to negate the CiREQ P PRI (1) signals for the other input child circuits. Finally, sinceonly three input child circuits 1006(i) are asserting their Ci REQ Psignals, the child request enumerator 1244 does not assert any of thelow priority Ci REQ P PRI (0) signals.

The round-robin number generator 1243 is essentially a counter that isenabled to increment by a round-robin control circuit 1245. Theround-robin control circuit 1245 receives the Ci REQ P child "Ci"requests parent signals from all input child circuits 1006(i) and theP3:P0 SEL [C3:C0] signals and enables the round-robin number generator1243 to increment under two circumstances. First, if the round-robincontrol circuit 1245 determines the condition of the Ci REQ P signalfrom the input child circuit 1006(i) whose index "i" corresponds to thevalue represented by the RND-RBN PRI round robin priority signals fromthe round-robin number generator 1243, and, if it is negated, it enablesthe round-robin number generator 1243 to increment. Second, theround-robin control circuit 1245 also enables the round-robin numbergenerator 1243 to increment if it determines that one of the P3:P0 SEL[Ci] signals, whose index "i" corresponds to the value represented bythe RND-RBN PRI round robin priority signals, is asserted.

The parent availability priority circuit 1241 receives the P3:P0 SEL ENparent select enable signals from the switching section 1201 andgenerates, for each, a set of Pi AVAIL PRI (3:0) parent "Pi"availability priority signals. The Pi AVAIL PRI (3:0) signals for each"Pi" comprise a plurality of priority signals identified by the mnemonicPi AVAIL PRI (x), each representative of a priority level. The Pi AVAILPRI (x) signals for each "Pi," and for descending values of "x"represents descending priority levels for the "Pi."

The parent availability priority circuit 1242 operates in a mannergenerally similar to that of the child request priority circuit, exceptthat the parent "Pi" having the highest priority is selected at random.The parent availability priority circuit 1242 includes a parentavailability enumerator 1246 that operates in response to the P3:P0 SELEN parent "Pi" select enable signal and RNDM NUM PRI random-numberpriority signals representing a random number. The parent availabilityenumerator 1246 operates in response to these signals in the same waythat the child request enumerator 1244 operates in response to the CiREQ P and RND-RBN PRI signals, respectively. The parent availabilityenumerator 1246 generates Pi AVAIL PRI (3:0) parent "Pi" availabilitypriority signals which identify the priority level for each "Pi," in amanner similar to the priorities for each "Ci" as identified by Ci REQ PPRI (3:0) signals produced by child request enumerator 1244.

As noted above, the parent "Pi" having the highest priority, asestablished by the parent availability priority circuit 1241, isselected at random. This selection is enabled by the random valuesrepresented by the RNDM NUM PRI random number priority signals. The RNDMNUM PRI signals are generated by a random number generator 1247, undercontrol of a control circuit 1250. The control circuit 1250 receives theCi REQ P request signals and the Pi SEL EN select enable signals andenables the random number generator 1247 to generate a new random numberin response to the coincidence of the conditions that (a) at least oneof the Ci REQ P signals is asserted and (b) at least two of the Pi SELEN select enable signals is asserted.

The child request/parent availability match circuit 1242 includes aconcentrator 1251 that receives all of the Ci REQ P PRI (3:0)) inputchild "Ci" requests parent priority signals for all input child circuits1006(i), and all of the Pi AVAIL PRI (3:0) parent "Pi" availabilitypriority signals for all "Pi" and generates in response thereto the PiSEL [C3:C0] select signals that control the switching section 1201. Ingenerating the Pi SEL [C3:C0] select signals, the concentrator 1251effectively determines the one of the Cx₃ REQ P PRI (3:0) signals havingthe highest priority and the one of the Py₃ AVAIL PRI (3:0) signalshaving the highest priority, and asserts the Py₃ SEL Cx₃ signal. Thissignal enables switching cell 1202(x₃,y₃), so that the data routermessage packet 30 from input child circuit 1006(x₃) to be coupledthrough the switching section 1201 and transmitted through output parentcircuit 1011(y₃). The concentrator 1251 does the same in connection withthe remaining Ci REQ P PRI (3:0) signals.

The concentrator 1251 also generates the P GRANTS Ci parent grants child"Ci" signals, which are used by the switch control section 1200 (FIG.11C-i) in generating the Ci REQ GRANTED signal. In particular, theconcentrator 1251 generates each P GRANTS Ci signal as essentially theOR of the Pj SEL Ci parent "Pj" selects child "Ci" signals for allparents "Pj."

As noted above, in one embodiment of data router 15 the fan-out, goingup the fat-tree defining the data router, may vary from level to level.At some levels, the fan-out is four, so that the data router node22(i,j,k) will have four input child circuits 1006(i) and four outputparent circuits 1011(i). At other levels, the fan-out is two, so thatthe data router node 22(i,j,k) will have four input child circuits1006(i) but only two active output parent circuits 1011(i). In thatembodiment, the data router nodes 22(i,j,k) are all implemented in thecircuitry, and so the circuitry for the other two output parent circuits1011(i) will be present, but rendered inactive by the node controlcircuit 1003 (FIG. 11A). In that case, the Pi SEL EN select enablesignals for the inactive output parent circuits 1011(i) will becontinually negated, and the parent availability enumerator 1246 willmaintain the Pi AVAIL PRI (3:0) signals in a negated condition. Since,in those levels, there will be fewer output parent circuits 1011(i) thatinput child circuits 1006(i), the round-robin priority maintained forthe input child circuits 1006(i) by the child request priority circuit1240 will ensure that the input child circuits 1006(i) will have be ableto transmit their data router message packets 30 on a reasonably equalbasis.

In addition, it will be appreciated that the random-number priorityprovided for the Pi AVAIL PRI (3:0) signals, under control of the randomnumber generator 1247, will ensure that the data router message packets30 going up the tree defining data router 15 are distributed randomlyamong the data router nodes 22(i,j,k). This minimizes the likelihood ofbottlenecks developing as data router message packets 30 are transmittedup the tree.

iii. Switch Cell 1202(i)

FIG. 11C-6 depicts the detailed circuit of a switch cell 1202(0,0) inthe switching section 1201 (FIG. 11C-1). All of the switch cells 1202are generally similar. With reference to FIG. 11C-6, the switch cell1202(0,0) receives the C0/SW FLIT input child "C0" flit to switchsignals in parallel on a bus 1260, the C0/SW FLY input child "C0" fly toswitch signal on a line 1261, and C0/SW FLOW flow from switch to inputchild "C0" signal on a line 1262. It will be appreciated that bus 1260and lines 1261 and 1262 extend through all switching cells 1202(0,j) inthe row of switching section 1201 associated with input child circuit1006(0). The switch cell also includes a bus 1263 that carries SWFLIT/C0 flit from switch to output child "C0" signals, a line 1264 thatcarries a SW FLY/C0 fly from switch to output child "C0" signal; and aline 1265 that carries a SW FLOW/C0 flow to switch from output child"C0" signal. Similarly, bus 1263 and lines 1264 and 1265 extend throughall switching cells 1202(i,0) in the column of switching cellsassociated with output child circuit 1007(0).

The switch cell 1202(0,0) also includes three gated driver circuits1270, 1271 and 1272, that are controlled in parallel by a CELL EN cellenable signal. Gated driver circuit 1270 has an input terminal connectedto bus 1260, and, when enabled, couples the C0/SW FLIT signals throughits output terminal onto the bus 1263 as the SW FLIT/C0 signals.Similarly, gated driver circuit 1271 has an input terminal connected toline 1261 and, when enabled, couples the SW FLY/C0 signal through itsoutput terminal onto to line 1264 as the SW FLY/C0 signal. Finally,gated driver 1272 has an input terminal connected to line 1262 and, whenenabled, it couples the SW FLOW/C0 signal through its an output terminalonto line 1265 as the C0/SW FLOW signal.

As noted above, the gated drivers 1270, 1271 and 1272 are controlled inparallel by the CELL EN cell enable signal. The CELL EN signal iscontrolled by a cell control circuit 1273. The cell control circuit 1273asserts the CELL EN signal upon receipt of an asserted C0 SEL [C0]signal from arbitration circuit 1211(0), and negates the CELL EN signalupon receipt of an asserted C0/SW RELEASE release to switch from inputchild "C0" signal on a line 1274. As with bus 1260 and lines 1261 and1262, the line 1274 extend through all switching cells 1202(0,j) in therow of switching section 1201 associated with input child circuit1006(0).

The control circuit 1273 includes a flip-flop 1275 whose data outputterminal provides the CELL EN cell enable signal. If the switch cell1202(0,0) is not enabled, the flip-flop 1275 is reset to negate the CELLEN signal. In that condition, a multiplexer 1276 is in condition tocouple the C0 SEL [C0] signal as a LAT SEL latch select signal to theflip-flop's direct set terminal. The CELL EN cell enable signal is alsocomplemented by an inverter 1280 to provide an asserted CELL(0,0) SEL ENcell (0,0) select enable signal, which is coupled by a multiplexer 1277as a LAT REL latch release signal to the direct reset terminal offlip-flop 1275 to maintain the flip-flop in a reset condition.

When the C0 SEL [C0] signal is asserted to enable switch cell 1202(0,0)the multiplexer 1276 asserts the LAT SEL signal, which sets theflip-flop 1276 to assert the CELL EN signal and enable gated drivers1270, 1271 and 1272. In addition, the asserted CELL EN signal enablesthe multiplexer 1276 to couple the CELL EN signal as the LAT SEL latchselect signal to maintain the flip-flop 1275 in the set condition. Inaddition, inverter 1280 negates the CELL (0,0) SEL EN signal.

The negated CELL (0,0) SEL EN signal enables multiplexer 1277 to couplea REL EN release enable signal from an AND gate 1281 signal as the LATREL latch release signal. The REL EN signal is generated by an AND gate1281. AND gate 1281 is energized, to assert the REL EN signal inresponse to the coincidence of the negation of the C0 SEL [C0] outputchild "C0" select input child "C0" signal and the assertion of the C0/SWRELEASE release from input child "C0" to switch signal. Thus, if eitherthe C0 SEL [C0] signal is asserted, indicating that the switch cell1202(0,0) is being selected by switch control section 1200 (FIG. 11C-1)or the C0/SW RELEASE signal is negated, indicating that the switch cellis being released, the AND gate 1281 maintains the REL EN signal in anegated condition.

When, after selection of the switch cell 1202(0,0), the input childcircuit 1006(0) asserts the C0/SW RELEASE signal at the end of a datarouter message packet 30, if the C0 SEL [C0] signal is negated the ANDgate 1281 is energized to assert the REL EN signal. When that occurs,the multiplexer 1277 asserts the LAT REL signal to reset the flip-flop1275 to negate the CELL EN cell enable signal, which, in turn, disablesgated drivers 1270, 1271 and. 1272. In addition, the negated CELL ENsignal enables the inverter 1280 to assert the CELL (0,0) SEL EN signal.In that condition, the multiplexer 1276 couples the negated C0 SEL [C0]signal as the LAT SEL to the direct set input terminal of flip-flop1275, and the multiplexer 1277 couples the asserted CELL (0,0) SEL ENsignal as the LAT REL signal to the flip-flop's direct reset terminal,to maintain the flip-flop 1275 in the reset condition.

It will be appreciated that, when the input child circuit 1006(0)asserts the C0/SW RELEASE signal at the end of a data router messagepacket 30, if the C0 SEL [C0] signal is asserted the AND gate 1281remains de-energized, to maintain the REL EN signal in a negatedcondition. When that occurs, the arbitration circuit 1211(0) ismaintaining selection of switch cell 1202(0,0) for the next data routermessage packet 30 from input child circuit 1006(0), and so the switchcontrol circuit. 1273 maintains the gated drivers 1270, 1271 and 1272enabled.

As noted above, when the flip-flop 1275 negates the CELL EN cell enablesignal at the end of a data router message packet 30, inverter 1280asserts the CELL (0,0) SEL EN select enable signal indicating that theswitch cell 1202(0,0) is in a released condition. As shown on FIG.11C-6, the CELL (0,0) SEL EN signal is coupled to an AND gate 1282,which receives corresponding CELL (0,j). SEL EN signals from the otherswitching cells 1202(0,3) in the same column of switching section 1201and generates the C0 SEL EN select enable signal when all of thesesignals are asserted. It will be appreciated that all of the switchingcells 1202(0,3) in that column will be connected to the same outputchild circuit 1007(0), and so if the C0 SEL EN signal is asserted, allof the switch cells 1202(0,j) in the column will be in the releasedcondition. As noted above, the assertion of the C0 SEL EN signal enablesthe arbitration circuit 1211(0) to perform select an input child orparent circuit 1006(i) or 1010(i) to begin transmitting a data routermessage packet 30 through a switching cell 1202(0,j) in the column tothe output child circuit 1007(0).

4. Output Child Circuit 1007(0)

FIG. 11D depicts a detailed diagram of output child circuit 1007(0).With reference to FIG. 11D, the output child circuit 1007(0) includes aswitch interface section 1300, a buffer section 1301 and an outputinterface section 1302. Generally, the switch interface section 1300receives flits of data router message packets 30 from the switchingsection, provided from an input child or parent circuit 1006(i) or1010(i), and couples them to the buffer section 1301. If the C0 OUT FLYsignal from child data router node 22(i,j,k) connected to the outputchild circuit 1007(0) is asserted, indicating that it can receive theflit, the output interface section 1302 receives the flit from thebuffer section and transmits it as C0 OUT FLIT signals. In addition, thebuffer section 1302 enables the switch interface section 1300 tomaintain the SW FLOW/C0 signal asserted, which indicates that the outputchild circuit 1007(0) is able to receive additional flits.

On the other hand, if the C0 OUT FLY signal becomes negated, indicatingthat the child data router node 22(i,j,k) is unable to receiveadditional flits, the output interface section enables the buffersection 1301 to begin buffering flits from the switch interface section.If the C0 OUT FLY signal is later asserted, the output interface section1302 enables the buffer section 1301 to resume providing flits for it totransmit, which the buffer section 1301 provides from the flits it hasbuffered. If at some point the buffer section 1301 has buffered a numberof flits, such that it becomes nearly full, it negates the SW FLOW/C0signal, which is coupled through the switch 1003 to the input child orparent circuit 1006(i) or 1010(i) supplying the flits. The negated SWFLOW/C0 signal disables the input child or parent circuit 1006(i) or1010(i). If the number of flits buffered in the buffer section 1301thereafter is reduced, the buffer section 1301 may thereafter enable theswitch interface section to assert the SW FLOW/C0 signal to, in turn,enable the input child or parent circuit 1006(i) or 1010(i) to resumesupply flits for transmission to the child data router node 22(i,j,k).

More specifically, the switch interface section 1300 includes a latch1303 that latches the SW FLIT/C0 flit from switch to output child "C0"signals and a latch 1304 that latches the SW FLY/C0 fly from switch tooutput child "C0" signal from the switch 1003 in response to thesuccessive ticks of the NODE CLK signal. In addition, the switchinterface section 1300 provides the SW FLOW/C0 signal to the switch1003. As described above in connection with FIG. 11C-6, the enabledswitch cell in switch 1003 couples the SW FLOW/C0 signal to theappropriate input child or parent circuit 1007(i) or 1011(i) as theCi/SW FLOW or the Pi/SW FLOW signal. In response, the input child orparent circuit couples signals which the enabled switch cell will coupleto the output child circuit 1007(0) as the SW FLIT/C0 and SW FLY/C0signal. Accordingly, it will be appreciated that, if the SW FLY/C0signal is asserted at a tick of the NODE CLK signal, the SW FLIT/C0signals represent a flit of a data router message packet 30 beingtransmitted by the source input child or parent circuit.

The SW FLIT/C0 signals are latched by latch 1303 at every tick of theNODE CLK signal. Latch 1303 provides, at its output terminals, LAT OUTFLIT latched output flit signals. The LAT OUT FLIT signals are coupledto one input terminal of a multiplexer 1306. If a first-in first-outbuffer 1305 is empty, it asserts a FIFO EMPTY signal, enabling the FIFO1306 to couple the LAT OUT FLIT signals as BUF OUT FLIT buffered outputflit signals to a gated driver 1313.

A latch 1312 latches the C0 OUT FLY signal from the child data routernode 22(i,j,k) at each tick of the NODE CLK signal. If the C0 OUT FLYsignal is asserted when the latch 1312 is enabled by the ticks of theNODE CLK signal, the latch 1312 maintains the OUT FLOW signal asserted.If the OUT FLOW signal, and if an EN enable signal is asserted by nodecontrol circuit 1004 (FIG. 11A) an AND gate asserts an EN OUT enable outsignal. The asserted EN OUT signal, in turn, enables the gated driver1313 to couple the BUF OUT FLIT signals as GATED OUT FLIT signals todata input terminals of a latch 1315. The latch 1315, in turn, latchesthe GATED OUT FLIT signals at each tick of the NODE CLK signal, andtransmits the latched signals as C0 OUT FLIT signals to the child datarouter node 22(i,j,k) connected thereto.

Contemporaneously, since the SW FLIT/C0 signals at that point representa flit of a data router message packet 30, the SW FLY/C0 signal is alsoasserted. The asserted SW FLY/C0 signals enables a latch 1304 to be setin response to the NODE CLK signal to assert a LAT OUT FLY latchedoutput fly signal. The asserted LAT OUT FLY signal is coupled to thebuffer section 1301, in particular to a push enable terminal of afirst-in first-out buffer (FIFO) 1305, to enable the FIFO 1305 to bufferthe flit represented by the LAT OUT FLIT signals.

The asserted LAT OUT FLY signal also energizes an OR gate 1307 to assertand OUT FLIT PRESET signal. The asserted OUT FLIT PRESENT signal enablesone input terminal of an AND gate 1311. The second input terminal of ANDgate 1311 is controlled by the OUT FLOW signal from the output interfacesection 1302, which at that point is asserted, energizing the AND gateto assert a POP signal. The POP signal enables the FIFO 1305 to transmitthe buffered flit as BUF FLIT signals. In addition, the FIFO 1305negates the FIFO EMPTY signal, enabling the multiplexer 1306 tothereafter couple the LAT OUT FLIT signals as the BUF OUT FLIT signals.

It will be appreciated that connecting multiplexer 1306 to selectivelycouple the LAT FLIT OUT signals or the BUF FLIT signals as the BUF OUTFLIT signals, in response to a FIFO EMPTY signal indicating whether theFIFO 1305 is empty, enables the signals from the switch 1003 to becoupled to the BUF OUT FLIT signals directly, through multiplexer 1306,without requiring them to be first buffered in FIFO 1305. This willeliminate any propagation delay through the FIFO 1305 if the FIFO 1305is empty.

On the other hand, if the C0 OUT FLY signal is negated, the latch 1312will negate the OUT FLOW. The disabled OUT FLOW signal disables AND gate1314 to negate the EN OUT signal, which, in turn, disables gated driver1313 from coupling the BUF OUT FLIT signals as the GATED OUT FLITsignals. Accordingly, the latch 1315 thereafter transmits negated C0 OUTFLIT signals to the child data router node 22(i,j,k) connected thereto.

The negated OUT FLOW signal also disables AND gate 1311, therebynegating the POP signal. Accordingly, while the latch 1304 is assertingthe LAT OUT FLY signal, FIFO 1305 is enabled to buffer the LAT OUT FLITsignals, which represent successive flits of one or more data routermessage packets 30, at successive ticks of the NODE CLK signal. If theFIFO 1305 becomes nearly full, it asserts a NR FULL signal, which iscomplemented by an inverter 1316 to negate the SW FLOW/C0 signal. Thenegated SW FLOW/C0 signal, in turn, disables the input child or parentcircuit 1006(i) or 1010(i) from continuing to couple flits thereto. Inaddition, the SW FLY/C0 signal will be negated, enabling the latch 1304to negate the LAT OUT FLY signal, which disables the FIFO 1305 fromlatching the LAT OUT FLIT signals.

When the child data router node 22(i,j,k) again asserts the C0 OUT FLYsignal, the latch 1312 asserts the OUT FLOW signal at the next tick ofthe NODE CLK signal. The asserted OUT FLOW signal energizes AND gate1314 to assert the EN OUT enable out signal, which, in turn, enablesgated driver 1313. The asserted OUT FLOW signal also enables one inputterminal of AND gate 1311. Since at this point the FIFO 1305 is notempty, the FIFO negates the FIFO EMPTY signal, which is complemented byinverter 1310 to energize OR gate 1307 to assert the OUT FLIT PRESENTsignal. The asserted OUT FLIT PRESENT signal also energizes the secondinput terminal of AND gate 1311, enabling it to assert the POP signal.While the POP signal is asserted, the FIFO 1305, at successive ticks ofthe NODE CLK signal, transmits BUF FLIT signals representing thesuccessive flits buffered therein as the BUF FLIT buffered flit signals.In addition, the negated FIFO EMPTY signals enables the multiplexer 1306to couple the BUF FLIT signals as the BUF OUT FLIT signals to gateddriver 1313. Since the gated driver is enabled, it couples the BUF FLITOUT signals as the GATED OUT FLIT signals to latch 1315.

When the contents of the FIFO 1305 has been reduced below apredetermined number of buffered flits, it will negate the NR FULLsignal, enabling, in turn, the inverter 1316 to assert the SW FLOW/C0signal. The asserted SW FLOW/C0 signal is coupled through the switch1003 to enable the appropriate input child or parent circuit 1006(i) or1010(i) to resume transmitting flits thereto.

When the FIFO 1305 thereafter becomes empty, it re-asserts the FIFOEMPTY signal, which conditions multiplexer. 1306 to resume coupling theLAT OUT FLIT signals as the BUF OUT FLIT signals.

D. Control Network 1. General

FIG. 12A is a general block diagram of a control network node 51 used inthe control network 14 described above, and FIGS. 12B through 12D-1comprise detailed block and logic diagrams of the control network node51. With reference to FIG. 12A, the control network node 51 includes aflick up control portion 1401, a flick down control portion 1402 and anup/down common portion 1403. The control network node 51 also includes adiagnostic network interface 1404, which provides an interface to thediagnostic network 16. In addition, the control network node 51 includesa clock buffer 1405 that receives the SYS CLK system clock signal fromthe clock circuit 17 (FIG. 1) and generates a NODE CLK node clock signalin response. In one particular embodiment, the clock buffer 1405comprises a buffer as described in aforementioned Hillis, et al., patentappn. Ser. No. 07/489,079, filed Mar. 5, 1990, entitled Digital ClockBuffer Circuit Providing Controllable Delay. It will be appreciated thatif all control network nodes 51 in a control network node group 50 (FIG.4B) are packaged together on, for example, a single integrated circuitchip, as is the case in one embodiment, the chip may be provided withone clock buffer 1405 which can provide the NODE CLK signal to allcontrol network nodes 51 in the control network node group 50.

Generally, the flick up control portion 1401 receives control networkmessage packets 60 from its child nodes and generates a control networkmessage packet 60 in response. If the control network node 51 is not aroot node, the flick up control portion 1401 transmits the generatedcontrol network message packet 60 to the parent control network node,thereby transmitting the packet up the tree defining the control network14. On the other hand, if the control network node 51 is a root node,the flick up control portion 1401 transmits the generated controlnetwork message packet 60 to the flick down control portion 1402 of thesame control network node 51 for transmission down the tree comprisingthe partition of the control network 14 of which the control networknode 51 is the root node. In addition, if the control network messagepackets 60 enable the node 51 to perform a scan operation, the flick upcontrol portion 1401 generates scan data and provides it to the flickdown control portion 1402.

The flick down control portion 1402 receives control network messagepackets 60 and generates control network message packets 60representative thereof for transmission to the child control networknodes, thereby transmitting message packets down the tree definingcontrol network 14. If the control network node 51 is not a root node,the flick down control portion 1402 uses control network message packets60 from the parent node. On the other hand, if the node 51 is a rootnode, it uses the control network message packets 60 from the flick upcontrol portion 1401 of the node 51. In addition, if the node 51 is nota root node, if the control network message packets 60 enable a scanoperation, the flick down control portion 1402 uses scan data providedby the flick up control portion 1402.

The common portion 1403 provides communications between the flick upcontrol portion 1401 and the flick down control portion 1402. The commonportion 1403 transmits control network message packets 60 and scan datafrom the flick up control portion 1401 to the flick down control portion1402. In addition, if the flick up control portion 1401 determines thatthe node 51 is to be a root node, the common portion 1403 notifies theflick down control portion 1402 and enables it to begin receivingcontrol network message packets 60 provided by the flick up controlportion 1401.

More specifically, the flick up control portion 1401 receives C(L) FLICKUP (4:0) signals representing successive flicks of control networkmessage packets 60 (FIG. 5) from a left child control network node 51and C(R) FLICK UP (4:0) signals representing successive flicks ofcontrol network message packets 60 from a right child control networknode 51 and, in response thereto, generates P FLICK UP (4:0) signals. Ifthe control network node 51 is not a root node, it transmits the P FLICKUP (4:0) signals to the parent control network node. On the other hand,if the control network node 51 is a root node, it buffers the P FLICK UP(4:0) signals in a packet buffer 1406 in the up/down common portion1403. In either case, if the received message packets 60 are of themultiple-source type ("multiple-source messages") initiating scanoperations, the control network node 51 also loads data generated inresponse to the message packets in a scan buffer 1410 in the commonportion 1403.

In addition, the flick up control portion 1401 also controlsestablishment or elimination of the control network node 51 as a rootnode. If it receives a control network message packets 60 that itdetermined to be of the single source message type ("single-sourcemessage") and configuration packet type, which establishes or eliminatesthe control network node 51 as a root node, the flick up control portion1401 conditions a root flag 1407 in the common portion 1403, and enablesassertion or negation of a ROOT UP signal. The root flag 1407 controlsselection by the flick down control portion 1402 of the source ofcontrol network message packets 60 received thereby.

The ROOT UP signal provided by the control network node 51 is receivedby the parent node. If the ROOT UP signal is asserted, indicating thatthe node 51 is a root node, the parent node internally couples to itsflick up control portion control network message packets 60 of theabstain type. In addition, the parent node is thereafter enabled totransmit control network message packets of the nil packet type to theflick down control portion 1402. It will be appreciated that, if theparent's other child node is not a root node, the parent node willcontinue transmitting to that other child node control network messagepackets 60 of message types representative of the packets it receivesfrom its parent, or representative of the packets its flick up controlportion receives if it is also a root node. If the control network node51 thereafter negates the ROOT UP signal, the parent node continuestransmitting, to the child node comprising a root node, control networkmessage packets of the nil packet type, until it is ready to begintransmission of a control network message packet 60 of another type.Thus, the parent node does not begin transmission of a control networkmessage packet 60 to a child node 51, whose status as a root node hasbeen eliminated, in the middle of the packet; instead, the parent nodewaits until the beginning of-the next packet after the end of the packetit is then transmitting.

The flick up control portion 1401 also generates, in response to receiptof a control network message packet 60 in which the all-fall-down bit 81(FIG. 5) is set, an AFD all-fall-down signal which may be coupled to thedata router 15.

The flick down control portion 1402 receives P FLICK DN (4:0) signalsor, if the control network node 51 is a root node, signals buffered inthe packet buffer 1406, and data buffered in the scan buffer 1410. Inresponse, the flick down control portion 1402 transmits C(L) FLICK DN(4:0) signals representing flicks of a control network message packet 60(FIG. 5) to a left child control network node 51 and C(R) FLICK DN (4:0)signals representing flicks of a control network message packet 60 to aright child control network node 51.

The flick down control portion 1402 also receives a C(L) ROOT UP signalfrom the left child node and a C(R) ROOT UP signal from the right childnode, which are used to control the types of message packets transmittedto the respective child nodes. If the C(x) ROOT UP signal ("x" referringto "R" or "L") is asserted, the respective child node is a root node,and in that case the flick down control portion 1402 transmits messagepackets of the NPAC nil packet type to the child node.

Otherwise, if the control network node 51 itself is a root node, theflick down control portion 1402 receives control network message packetsfrom the packet buffer 1406 and uses them to generate message packetsfor transmission to the child nodes. If the message packet received bythe flick down control portion 1402 is of the single-source or idletypes, it transmits it to the child nodes that are not asserting theirC(x) ROOT UP signals. On the other hand, if the received message packetis a multiple-source message, the flick down control portion 1402 usesthe message packet from the packet buffer 1406 in generating multiplesource message packets for transmission to the child nodes that are notasserting their C(x) ROOT UP signals.

Both the flick up control portion 1401 and the flick down controlportion 1402 also exchange flow control information over a link 1411,that they provide in the scan flow bits 72(i) (FIG. 5) in the controlnetwork message packets 60 that they transmit. In addition, each portion1401 and 1402 provides the flow control information from the controlnetwork message packets 60 that they receive to the other portion, whichuses the information in regulating the transmission of packets 60. Therespective portions remain disabled, until the other portion receivesflicks in which the scan flow bit 72(i) is clear, after which theyresume transmission.

In the control network nodes 51 comprising the control network 14, theflow control is on a control network message packet basis, so that if acontrol network node 51 begins transmission of a control network messagepacket 60 to another node, the receiving node receives the entirepacket. Thus, if the receiving node transmits a control network messagepacket 60 to the transmitting node in which the scan flow bit 72(i) isset, to disable transmission by the transmitting node, the transmittingnode will continue transmission of the packet it is currentlytransmitting and will thereafter become disabled.

In addition, the flow control only controls transmission of messagepackets 60 of the multiple source type. If a disabled node is totransmit a message packet of a type other than multiple source, ittransmits it in the same manner as if it were not disabled. On the otherhand, if the is the next message packet to be transmitted by a disablednode is of the multiple source type, it delays transmission and insteadtransmits message packets of the idle type. When the receiving node atsome point later indicates that it can resume reception, after itfinishes transmission of the current idle message packet, thetransmitting node transmits the delayed multiple source message packet60.

The details of several elements depicted on FIG. 12A will be describedin connection with FIGS. 12B through 12D-1. In particular, the flick upcontrol portion 1401 will be described in connection with FIGS. 12Bthrough 12B-4D. The details of the root flag 1407 and control circuitrytherefor will be described in connection with FIG. 12C. Finally, thedetails of the flick down control portion will be described inconnection with FIGS. 12D through 12D-1.

2. flick up control portion 1401 i. General

FIGS. 12B depicts a general block diagram of the flick up controlportion 1401, and FIGS. 12B-1 through 12B-4D depict detailed block andlogic diagrams of the flick up control portion 1401. With reference toFIG. 12B, the flick up control portion 1401 includes a child (left)receiver/buffer 1420(L) which receives the C(L) FLICK UP (4:0) signalsfrom the left child control network node 51 and a child (right)receiver/buffer 1420(R) which receives the C(R) FLICK UP (4:0) signalsfrom the right child control network node 51. The receiver/buffers,which are generally identified by reference numeral 1420(x) aregenerally similar, and will be described below in connection with FIGS.12B-1 through 12B-1G.

Generally, each receiver/buffer 1420(x) receives the C(x) FLICK UPsignals from the respective left or right child control network node 51representing successive flicks of control network message packets 60. Inresponse, the buffer/receivers 1420(x) provides SEL INP DATA (x)selected input data signals to a flick up data processor 1421 and to anup output packet assembler 1422, and INP TAG (x) input tag signals to atag processor 1423. In addition, the receiver/buffers 1420(x) provide(X) INP STA/CTRL left and right status/control signals to an up controlcircuit 1424 identifying the timing of receipt of the respective controlnetwork message packets 60, and the respective types of message packetsbeing received.

In response to the (X) INP STA/CTRL status/control signals identifyingthe types of message packets being received by the respectivebuffer/receivers 1420(x), the up control circuit 1424 provides OUT SELoutput select signals that identifies a message type to be generated bythe up output packet assembler 1422. The up output packet assembler 1422begins generating P FLICK UP signals for a control network messagepacket 60 of the type identified by the OUT SEL signals. A flick latch1430 latches P FLICK UP signals at each tick of the NODE CLK signal, andtransmits the latched signals to its parent node as FLICK OUT (P) flickout to parent signals.

In addition, as the up output packet assembler 1422 generates P FLICK UPsignals representing the sequential flicks of the control networkmessage packet 60, the OUT SEL signals enable it to use the SEL INP DATA(x) signals from the left or right child buffer/receiver 1420(x) or PROCFLICK (LIP) DATA processed flick data signals form the flick (up) dataprocessor 1421. The PROC FLICK (UP) DATA signals represent the sum,logical OR, logical XOR and maximum of the SEL INP DATA (x) signals, asgenerated by an adder 1425, an OR circuit 1426, an XOR circuit 1427 anda comparator 1428, respectively. The OUT SEL output select signals mayenable the up output packet assembler 1422 to use the PROC FLICK (UP)DATA signals from one of these circuits 1425 through 1428 in a controlnetwork message packet 60 it is currently generating.

It will be appreciated that the PROC FLICK (UP) DATA signals from theflick (up) data processor 1421 are used in connection with controlnetwork message packets 60 of the multiple-source message type and scanand reduce packet types. The flick (up) data processor 1421 provides thecombination, as called for by the scan or reduce operation initiated bythe received message packets 60, of the data in the data nibbles 70(i)(FIG. 5). Thus, the up control circuit 1424 enables the up output packetassembler 1422 to use the PROC FLICK (UP) DATA signals in a controlnetwork message packet 60 if the packets 60 received from the childrenare of the appropriate message and packet types.

The particular type of control network message packet 60 identified bythe OUT SEL output select signals from the up control circuit 1424depend on the types of packets 60 received by the receiver/buffers1420(x) and by a selected priority arrangement. In one particularembodiment, if both receiver/buffers 1420(x) receive message packets 60of the same message type, the OUT SEL signals will generally enable theup output packet assembler 1422 to generate P FLICK UP signals for acontrol network message packet 60 of that type. If, for example, thereceiver/buffers 1420(x) both receive single-source messages, the upcontrol circuit 1424 will enable the up output packet assembler 1422 totransmit a single-source message.

In that case, if the message packet is of the configuration packet type,the up control circuit 1424 will also enable the up output packetassembler to use a signal, provided by the comparator 1428 identifyingwhich of the SEL INP DATA (L) and SEL INP DATA (R) selected input datasignals represents the larger value, and uses the identified selectedinput data signals in the outgoing control network message packet 60.Thus, the root height value provided in data nibbles 70(0) and 70(1) isthe maximum of the values in the control network message packets 60received by the receiver/buffers 1420(x). However, if the root heightvalues in the received message packets differ, an error has occurred inthe system 10. In that case the up control circuit 1424 generates an ERRerror signal, which enables the flick down control portion 1422 to setthe S ERR software error bit 76 in the control network message packets60 it is transmitting.

Similarly, if the control network message packets 60 being received bythe receiver/buffers 1420(x) are of the multiple-source message type,and of the same packet type, the up control circuit 1424 generates OUTSEL signals that enable the up output packet assembler 1422 to generateP FLICK UP signals representing a control network message packet 60 ofthe same message and packet type. The up control circuit 1424 enablesthe up output packet assembler 1422 to include, in the packet dataportion 62 (FIG. 5), PROC FLICK (UP) DATA processed flick up datasignals from the flick (up) data processor 1421. The particular one ofcircuits 1425 through 1428 whose PROC FLICK (UP) DATA signals are useddepends upon the particular type of scan or reduce operation is enabledby the received control network message packets 60. In addition, the upcontrol circuit 1424 generates SCAN BUF WE scan buffer write enablesignals that enable the scan buffer 1410 to load the SEL INP DATA (L)signals from the receiver/buffer 1420(1) representing the successivedata nibbles 70(i).

On the other hand, if one receiver/buffer 1420(x) receives asingle-source message, and the other receiver/buffer 1420(x') receiveseither a multiple-source message or a control network message packet 60of the abstain message type (an "abstain message") or the idle messagetype (an "idle message"), the up control circuit 1420 generates OUT SELsignals that enable the up output packet assembler 1422 to transmit asingle-source message. In that operation, the up output packet assembleruses the SEL INP DATA (x) signals from the receiver/buffer 1420(x) thatreceived the single-source message. Thus, a single-source message takespriority over multiple-source, abstain or idle messages. If thereceiver/buffer 1420(x') receives a multiple-source message, the upcontrol circuit 1424 also enables it to buffer the packet. As will bedescribed below, the buffered packet 60 will be used in connection witha multiple-source message when received by the first receiver/buffer1420(x), that is, the receiver/buffer that received the single-sourcemessage.

In addition, if a receiver/buffer 1420(x) is buffering a multiple-sourcemessage, the up control circuit 1424 generates FLOW CTRL (DN) flowcontrol signals that are coupled through the line 1411 (FIG. 12A) to theflick down control portion 1402. In response, the flick down controlportion sets the scan flow bits 72(i) (FIG. 5) of the control networkmessage packets 60 of the particular left or right child node connectedto the receiver/buffer 1420(x), to disable the child node fromtransmitting additional message packets 60. When-the buffered packet 60is thereafter used, the up control circuit 1424 generates FLOW CTRL (DN)signals that thereafter enable the flick down control portion to clearthe scan flow bits 72(i) of the packets 60 transmitted to that childnode, enabling the child node to resume transmitting message packetsthereto.,

Alternatively, if one receiver/buffer 1420(x) receives or is buffering amultiple-source message and the other receives an abstain message, theup control circuit 1424 generates OUT SEL output select signals thatenable the up output packet assembler to generate P FLICK UP signalsfrom the SEL INP DATA (x) signals representing the received multiplesource message. The abstain message indicates that the leaf 21 (FIGS. 4Aand 4B) comprising the source of the message is abstaining from theoperation initiated by the multiple-source message received or bufferedby the other receiver/buffer 1420.

In addition, if a receiver/buffer 1420(x) receives a multiple-sourcemessage and the other receiver/buffer 1420(x') receives an idle message,the up control circuit 1424 enables the receiver/buffer 1420(x) tobuffer the multiple-source message. When the other receiver/buffer1420(x') thereafter receives an abstain or multiple-source message, theup control circuit 1424 enables the receiver/buffer 1420(x) to providethe buffered message as SEL INP DATA (x) signals to be used by the flickup data processor 1421 and up output packet assembler 1422 in generatinga control network message packet 60 for transmission to the parentcontrol network node 51. In that case, the buffered multiple-sourcemessage is used in the same manner as if it were being receivedcontemporaneously with receipt by the receiver/buffer 1420(x') of theabstain or multiple-source message being received thereby. Similarly, ifboth receiver/buffers 1420(x) are buffering multiple-source messages,and if both receive idle messages, the up control circuit enables themto provide the buffered messages as SEL INP DATA signals to be used bythe flick up data processor 1421 and up output packet assembler 1422 ingenerating a control network message packet 60 for transmission to theparent control network node 51.

Finally, if both receiver/buffers 1420(x) receive control networkmessage packets 60 of the idle, abstain or NPAC (nil packet) messagetypes, and if they are not buffering multiple-source message packets,the up control circuit 1424 enables the up output packet assembler 1422to transmit a control network message packet 60 to the parent controlnetwork node 51 of the same message type.

In generating P FLICK UP signals representing the sequential flicks of acontrol network message packet 60, the up output packet assembler usesPROC TAG (UP) processed tag signals from the tag processor 1423. the tagprocessor 1423 receives the INP TAG (x) signals from thereceiver/buffers 1420(x) and generates PROC TAG (UP) processed tagsignals that the up output packet assembler 1422 uses in generating thetag bit in each flick. The PROC TAG (UP) signals represents the logicalAND and logical OR of the INP TAG (x) input tag signals. Depending onthe particular flick and tag bit being transmitted, the up controlcircuit 1424 may enable the up output packet assembler 1422 to transmitthe signal representing the logical AND or the signal representing thelogical OR.

For some flicks, particularly those containing the scan flow bits 72(i)and the overflow bit, the up control circuit 1424 enables the up outputpacket assembler 1422 to use signals from other circuits in generatingthe tag bits. In particular, when the up output packet assembler 1422 isgenerating P FLICK UP signals representing flicks in which the tag bitscomprise the scan flow bits 72(i), the up control circuit 1424 enablesthe up output packet assembler 1422 to use FLOW CONTROL (UP) signals ingenerating the bits. The up output packet assembler 1422 receives theFLOW CONTROL (UP) signals from the flick down control circuit 1402through the flow control link 1411 in the common control portion 1403.

In addition, when the up output packet assembler 1422 is generating theflick containing the scan overflow bit 80 (FIG. 5), the up controlcircuit 1424, if it enabled the up output packet assembler 1422 totransmit PROC FLICK (UP) DATA signals from the adder circuit 1425 in thedata nibbles 70(i), also enables it to use a CRY IN carry in signal andan OVFL overflow signal from the flick up data processor 1421. The CRYIN and OVFL signals represent carry and overflow signals generated bythe adder circuit 1425, and if asserted indicates a carrry and overflow,respectively, in the sum provided by the adder circuit 1425.

The up control circuit 1424 provides several additional signals forcontrolling the operation of the flick up control portion 1401. Inparticular, the up control circuit 1424 generates timing controlsignals, identified as UP RCV ST (12:0) signals, which are timingsignals similar to the XMIT (12:0) transmit flick and RCV (12:0) receiveflick signals generated by the control network interface 204 in thenetwork interface 202 (FIG. 8) as described above. In addition, the upcontrol circuit 1424 generates an UP RCV RESET receive reset timingsignal. In particular, the up control circuit 1424 receives the SEL INPDATA (x) selected input data signals from the receiver/buffer circuits1420(x) and, if they indicate that flicks being received represent NPACnil packet message packets 60, the up control circuit 1424 asserts theUP RCV RESET timing signal.

On the other hand, if the SEL INP DATA (x) signals indicate that atleast one receiver/buffer is receiving the first flick of a controlnetwork message packet 60, up control circuit 1424 begins assertingsuccessive ones of UP RCV ST (12:0) up receive state signals thatidentify successive receive states. The UP RCV ST (12:0) signalsrepresent thirteen signals, identified by the mnemonic UP RCV ST 0through UP RCV ST 12, which the up control circuit 1424 successivelyasserts, at successive ticks of the NODE CLK timing signal. Generally,the successive UP RCV ST "i" signals ("i" is an integer between zero andtwelve) are asserted in synchronism with the receipt by thereceiver/buffers 1420(x) of corresponding ones of the thirteen flicks ofa control network message packet 60. The up RCV ST 0 through LIP RCV ST12 signals are used to other circuitry on the flick up control portion1401.

In addition, the up control circuit 1424 depicted on FIG. 12B generatesa ROOT UP LAT root up latch signal that enables conditioning of the rootflag 1407 (FIG. 12A) which, in turn, controls the condition of the ROOTUP signal. If the SEL INP DATA (x) signals represent a single-sourcemessages of the configuration type, and if the root height valuecorresponds to the level and sub-level of the control network node 51,the up control circuit 1424 asserts the ROOT UP LAT signal to set theroot flag 1407. On the other hand, if the SEL INP DATA (x) signalsrepresent a single-source messages of the configuration type, and if theroot height value is greater than the level and sub-level of the controlnetwork node 51, the up control circuit negates the ROOT UP LAT signalto enable clearing of the root flag 1407. When the root flag 1407 isset, the ROOT UP signal is asserted, which energizes a write enableterminal of the packet buffer 1406, enabling the buffer 1406 to bufferthe P FLICK UP signals from the up output packet assembler 1422. Thepacket buffer 1406 provides BUF P FLICK UP buffered parent flick upsignals to the flick down control portion 1402.

As described above, the flick latch 1430 latches the P FLICK UP signalsat each tick of the NODE CLK signal. In addition, while the ROOT UPsignal is asserted, the packet buffer 1406 latches the P FLICK UPsignals at each tick of the. NODE CLK signal. In one particularembodiment, the right and left child receiver/buffers 1420 also latchthe respective C(x) FLICK UP signals from their respective child nodesat each tick of the NODE CLK signal. In that embodiment, the delay ofthe signals from the receiver/buffers 1420 to the flick latch 1430 andpacket buffer 1406 is one tick of the NODE CLK signal. That is, unless amessage packet 60 is being buffered by a receiver/buffer 1420(x), theflick latched by the receiver/buffers 1420 is processed by the flick(up) data processor 1421, tag processor 1423, and up output packetassembler 1422 during the time between successive ticks of the NODE CLKsignal. In that embodiment, the up output packet assembler 1422maintains information as to the type message and packet it istransmitting for use by other circuits shown on FIG. 12B.

FIGS. 12B-1 through 12B-3 depict details of some of the circuits in theflick up control portion 1401. In particular, FIG. 12B-1 depicts adetailed block diagram of the child (left) receiver/buffer 1420(L), andFIGS. 12B-1A through 12B-1G depicts details of circuits in the child(left) receiver buffer 1420(L). The child (right) receiver/buffer1420(R) is substantially similar to the child (left) receiver buffer1420(L) and will not be described in detail. FIG. 12B-2 depicts detailsof the flick (up) data processor 1421. Finally, FIG. 12B-3 depicts adetailed block diagram of the output packet assembler.

i. Child (Left) Receiver/Buffer 1420(L)

FIG. 12B-1 depicts a detailed block diagram of the child (left)receiver/buffer 1420(L), and FIGS. 12B-1A through 12B-1G depicts detailsof circuits in the child (left) receiver buffer 1420(L). With referenceto FIG. 12B-1, the child (left) receiver/buffer 1420(L) includes asource data selector 1440 that, under control of SRC DATA SEL sourcedata select signals from an input source identifier circuit 1441,selects from among several signal sources to provide the SEL INP DATA(L) selected input data (left) signals.

The source data selector 1440 can select among number of signals inputthereto. In particular, the source data selector 1440 may select C(L)FLICK UP (3:0) LAT latched child (left) flick up signals, as latched byan input latch 1443. The input latch latches the C(L) FLICK UP (4:0)signals received from the child control network node 51 at each tick ofthe NODE CLK signal. In addition, the C(L) FLICK UP (4:0) signals arecoupled to a check circuit 1444 that, in response to the NODE CLK andRCV ST (12:0) receive state signals, performs a check operation inconnection with the checksum in field 63 (FIG. 5) of the control networkmessage packet 60 being received to verify proper receipt of the packet60.

The source data selector 1440 may also select PMCL parked multi-sourceleft child signals from a left park buffer 1442. As noted above, thereceiver/buffer 1420(L) may buffer data from a multiple source messageif the other receiver/buffer is not then receiving a multiple-sourcemessage whose data is to be used in the required mathematical operation.The left park buffer 1442 provides this facility in the receiver/buffer1420(L).

As also noted above, in connection with a scan backward operationenabled by a multiple-source message, the left and right inputs to eachcontrol network node 51 are effectively reversed. To accommodate that,the source data selector 1440 of the left receiver/buffer 1420(L) canalso receive C(R) FLICK UP (3:0) LAT latched flick up signals from theright child node, and also PMCR parked multi-source right child signalfrom a right park buffer (not shown). Finally, if both child nodes areroot nodes, the source data selector 1440 may couple ABSN abstainsignals which have the encoding corresponding to that in the messagetype field 64 of a control network message packet 60.

The receiver/buffer 1420(L) also includes a tag input selector 1445that, under control of the input source identifier, selects among anumber of signal sources as the SEL INP TAG (L) selected input tagsignal. The tag input selector 1445 also may couple the C(L) FLICK UP(4) LAT latched left child flick up signal from the input latch 1443, orthe corresponding C(R) FLICK UP (R) LAT signal received by the receiverbuffer 1420(R) from the right child. It will be appreciated that, if theinput source identifier conditions the SRC DATA SEL source data selectsignals to enable the source data selector 1440 to couple the C(x) FLICKUP (3:0) LAT signals ("x" is "L" or "R") as the SEL INP DATA (L)signals, it will also condition the SRC TAG SEL signals to enable thetag in selector 1445 to couple the C(x) FLICK UP (4) LAT signals as theSEL INP TAG (L) signal.

In addition, the tag input selector can select a POVL parked overflowleft signal from a park overflow buffer 1446 in the left childreceiver/buffer. 1420(L), or a POVR parked overflow right signal from acorresponding buffer in the right child receiver/buffer 1420(R). Thepark overflow buffer 1446 buffers the scan overflow bit 80 of a receivedcontrol network message packet 60, whose data is parked in the parkbuffer 1442. Thus, if the input source identifier 1441 enables thesource data selector 1440 to couple PMCx ("x" is "L" or "R") parkedmulti-source signals from a park buffer 1442, or the correspondingbuffer in the right child receiver/buffer 1420(R), it will also enablethe tag in selector 1445 to use the POVx parked overflow signal as theSEL INP TAG (L) selected input tag signal for the scan overflow bit. Itwill be appreciated that, if the input source identifier conditions theSRC DATA SEL source data select signals to enable the source dataselector 1440 to couple the PMCx signals ("x" is "L" or "R") as the SELINP DATA (L) signals, it will also condition the SRC TAG SEL signals toenable the tag in selector 1445 to couple the POVx signals as the SELINP TAG (L) signal for the scan overflow bit.

The input source identifier 1441 uses a number of signals in generatingthe SRC DATA SEL source data select signals. In particular, an inputpacket type decoder 1447 receives the C(L) FLICK UP (3:0) LAT from theinput latch 1443, the C(R) FLICK UP (3:0) I.,AT signals from the rightchild receiver/buffer 1420(R) and the UP RCV ST 0 signal. It will beappreciated that the C(x) FLICK UP (3:0) LAT signals, when the UP RCV ST0 signal is asserted, identifies the particular message type of theincoming control network message packet 60. The input packet typedecoder 1447 generates a series of signals, including signals Cx/IDLEidle, Cx/SS single-source, Cx/MS multiple-source, Cx/ABS abstain, andCx/NPAC nil packet ["x" identifies "L" (left) and "R" (right)]. Thesesignals identify the particular type of control network message packet60 being received from each of the left and right child nodes. The inputsource identifier 1440 uses these signals, along with the UP RCV ST(12:0) up receive state signals, to determine whether the source dataselector 1440 should couple signals from an input latch, such as latch1443, or from a park buffer 1442, or the ABSN signals, as the SEL INPDATA (L) selected input data signals.

The input source identifier 1441 uses an OUT PKT MS output packetmultiple-source signal and an OUT PKT SCF/RED output packet scanforward/reduce signal, both of which are generated by the up outputpacket assember 1422, to determine whether the source data selector 1440is to couple signals from the right or left child, or right or left parkbuffer, as the SEL INP DATA (L) signals. As described above, the upoutput packet assembler 1422 generates signals providing information asto the packet 60 being transmitted thereby, of which the OUT PKT MS andOUT PKT SCF/RED signals are two. The OUT PKT MS and OUT PKT SCF/REDreduce signals, when asserted, indicate that the P FLICK UP signalsgenerated by the up output packet assember 1422 comprise a multiplesource control network message packet 60, and that the packet typeenables a scan forward or reduce operation. In that case, the inputsource identifier 1441 enables the source data selector 1440 to coupleeither the C(L) FLICK LIP (3:0) LAT signals or the PMCL signals as theSEL INP DATA (L) signals. Contemporaneously, the input source identifier1441 generates SRC TAG SEL source tag select signals that enable the tatinput selector 1445 to couple either the C(L) FLICK UP (4) LAT signal orthe POVL signal as the SEL INP TAG (L).

On the other hand, if the OUT PKT MS signal is asserted, but the OUT PKTSCF/RED signal is negated, the control network message packet 60 is ofthe multiple-source message type, and indicating a scan backwardoperation. In that case, the input source identifier generates SRC DATASEL signals that enable the source data selector 1440 to couple eitherthe C(L) FLICK UP (3:0) LAT signals latched by the right childreceiver/buffer 1420(R), or the PMCR signals provided by the park bufferof the right child receiver/buffer 1420(R) as the SEL INP DATA (L) leftselected input data. Thus, if the control network message packet 60 isof the multiple-source message type, and enabling a scan backwardoperation, the right and left child receiver/buffers 1420 interchangethe C(x) FLICK UP signals received thereby, as described above.

Furthermore, if the C(L) ROOT UP signal from the child node connectedthereto is asserted, indicating that the child node is a root node, theinput source identifier 1441 generates SRC DATA SEL signals to enablethe source data selector to couple ABSN abstain signals as the SEL INPDATA (L) signals. Thus, if the child node is root node, the childreceiver/buffer 1420(1) internally provides abstain packets, regardlessof the types of control network message packets 60 directed thereto bythe child node.

In any case, the input source identifier 1441, while the RCV ST 11 andRCV ST 12 signals are asserted, indicating that the global informationfield 71 and checksum field 63 of the control network message packet 60are being received, generates SRC DATA SEL source data select signals toenable the source data selector 1440 to couple the C(L) FLICK UP (3:0)LAT signals as the SEL INP DATA (L) signals. With respect to the globalinformation field 71, regardless of the message types of the messagesincluding the field 71, the signals representative of the field are ORedtogether with corresponding signals from the right child receiver/buffer1420(r). The resulting signals are transmitted in control networkmessage packets 60 up the tree comprising the control network 14 to theroot node, which, in turn, broadcasts them in control network messagepackets 60 transmitted down the tree to the leaves 21 in the partition.Thus, the global bits may be used to provide information concerning, forexample, synchronization of the operations of the processing elements11, regardless of the types of control network message packets 60 beingtransmitted up and down the tree defining the control network 14.

The left child receiver/buffer 1420(L) also includes control and statuscircuits 1450 and 1451 for controlling the park buffer 1442 and parkoverflow buffer 1446, respectively. The park buffer control and statuscircuit also receives a number of signals and generates, in response, PKBUF (L) CTRL left park buffer control signals, a PK BUF SRC SEL (L) leftpark buffer source select signal, and PK BUF (L) ST left park bufferstatus signals. The PK PUB SRC SEL (L) signal enables a multiplexer 1452to couple either the SEL INP DATA (L) selected input data signalsrepresenting a control network message packet 60 from the source dataselector 1440 to data input terminals of the park buffer 1442. The PKBUF (L) CTRL signals enable the signals from the multiplexer 1452 to bebuffered in the park buffer 1442, to enable it to buffer a controlnetwork message packet 60. In particular, the contents of park buffer1442 comprise the low-order four bits of each flick of the controlnetwork message packet 60. The PK BUF (L) ST left park buffer statussignals indicate whether the park buffer 1442 is buffering a controlnetwork message packet 60, and, if so, whether the segment bit 77 is setor cleared.

The only tag information buffered by the receiver/buffer 1420(L)indicates the condition of the segment bit 77 and the scan overflow bit80. It will be appreciated that these tag bits 77 and 80 are associatedwith, or provide information as to the processing of the data in thedata nibbles 70(i) of the control network message packet 60. On theother hand, the other tag bits of a control network message packet 60provide control information to control the flow of control networkmessage packets 60 through the control network 14, or to control theall-fall-down operations of the data router 15.

In any event, if the input packet type decoder 1447 is asserting theCL/MS signal and either the CR/SS or CR/IDLE signal, the control networkmessage packet 60 represented by the SEL INP DATA (L) signals is acandidate to be buffered in the park buffer 1442. In that case, thepacket 60 received by the left child receiver/buffer 1420(L) is amultiple-source message, and the right child receiver/buffer 1420(R) isnot receiving a multiple-source message to be used in connectiontherewith. If the park buffer in the right child receiver/buffer 1420(R)is not buffering a packet 60 which can be used with the multiple-sourcemessage being received by the left child receiver/buffer 1420(L), thepark buffer control/status circuit 1450 will condition the PK BUF SRCSEL (L) park buffer source select signals to enable multiplexer 1452 tocouple the SEL INP DATA (L) signals representing the packet 60 to thepark buffer 1442, and the PK BUF CTRL (L) park buffer control signals toenable the park buffer to buffer the packet 60.

On the other hand, if the right child receiver/buffer 1420(R) isreceiving a multiple-source message packet while (a) the park buffer1442 of the left child receiver/buffer 1420(L) is buffering amultiple-source message packet, and (b), the left child receiver/buffer1420(L) is not receiving a control network message packet 60 of a type,such as a single-source message packet having higher priority oftransfer through the control network 14, the park buffer control/status1450 enables the input source identifier 1441 to, in turn, enable thepark buffer control/status 1450 to transmit the PMCL signalsrepresentative of the successive flicks of the multiple-source controlnetwork message packet 60. The input source identifier 1441 of the leftchild receiver 1420(L), or the corresponding circuit of the right childreceiver 1420(R), enables the respective source data selector 1440 tocouple the PMCL signals as the SEL INP DATA (L) signals, if theoperation enabled is a scan forward or reduce operation on the one hand,or a scan backward operation on the other hand.

As noted above, the PK BUF (L) ST left park buffer status signals fromthe park buffer control/status circuit 1450 provide information as towhether the segment bit 77 was set in the message packet 60 buffered inpark buffer 1442. The left park buffer control/status circuit 1450receives the SEL INP TAG (L) selected left input tag signal from the taginput selector 1445 for this.

The park overflow buffer status/control circuit 1451 uses the PK BUF (L)ST left park buffer status signals, the UP RCV ST 10 receive timingsignal, and the OUT PKT MS output packet multiple-source signal, andgenerates a PK OVFL SRC SEL (L) park overflow source select and PK OVFLBUF CTRL park overflow buffer control signals to control the parkoverflow buffer 1446 and an input multiplexer 1453. If the PK BUF (L) STleft park buffer status signals indicate that the park buffercontrol/status circuit 1450 is enabling buffering of a control networkmessage packet 60, the park overflow buffer status/control circuit 1451enables the multiplexer 1453 to couple the SEL INP TAG (L) signalrepresenting the scan overflow bit 80 to be buffered in the parkoverflow buffer 1446.

With this background, the details of the left child receiver/buffer1420(L) will be briefly described in connection with FIGS. 12B-1Athrough 12B-1G. FIGS. 12B-1A and 12B-1B depict details of the sourcedata selector 1240 and tag input selector 1445, respectfully, along withrespective portions of the input source identifier 1441. FIGS. 12B-1Cthrough 12B-1E depict details of the park buffer 1442, multiplexer 1452and the park buffer control/status circuit 1450. FIG. 12B-1F depictsdetails of the park overflow buffer 1446, multiplexer 1453 and the parkoverflow buffer status/control circuit 1451. Finally, FIG. 12B-1Gdepicts details of a segment bit latch, in the park buffercontrol/status circuit 1450, which provides a SEG L left segment signalindicating the condition of the received or buffered segment bit of thecontrol network message packet 60 defined by the successive SEL INP DATA(L) selected input data signals. The SEG L signal is buffered in thescan buffer 1410.

Since the operation of the circuits depicted on FIGS. 12B-1A through12B-1G will, in view of the description in connection with FIGS. 12A and12B-1, be apparent to those skilled in the art, they will not bedescribed in detail. In any event, with reference to FIG. 12B-1A, thesource data selector 1440 includes to multiplexers 1460 and 1461. Themultiplexer 1460 select among the C(x) FLICK UP (3:0) LAT latched leftand right child flick up signals, the PMCR signals from the park bufferof the right child receiver/buffer 1420(R), and SEL PMCL/ABSN signalsfrom the multiplexer 1461. The multiplexer 1461 selects among the PMCLsignals from the park buffer 1442 of the left child receiver/buffer1420(L) and ABSN abstain signals encoded to conform to the encodingwhich, in message type field 64 of a control network message packet 60,identifies the abstain message type.

The multiplexer 1461 is controlled by an abstain/park buffer selectcontrol circuit 1462, and the multiplexer 1460 is controlled by acontrol circuit 1463. The abstain/park buffer select control circuit1462 enables the multiplexer 1461 to couple the ABSN signals to themultiplexer 1460 if an OR gate 1464 is energized, which occurswhen-either the UP RCV ST RST or the UP RCV ST 12 receive timing signalis asserted, or when an AND gate 1465 is energized. AND gate 1465 isenergized in response to the coincidence of the UP RCV ST 0 signal isasserted and both the Cx/NPAC signals are asserted. The last conditionoccurs if both the child nodes are root nodes.

The control circuit 1463 includes several sections, including amultiplexer control circuit 1466, a left/right select enable circuit1467, and a message packet priority circuit 1470. The multiplexercontrol circuit 1466, in turn, includes three parts. An output enablepart 1471 generates a RCV DATA OUT EN (L) left received data outputenable signal, which enables or disables output by the multiplexer 1460.A left/right select circuit 1472 enables the multiplexer 1460 togenerally selectively couple either signals received from the left childor the right child as the SEL INP DATA (L) selected left input datasignals.

More specifically, the left/right select circuit 1472 provides ahigh-order SEL LEFT LAT select left latched signal, that, when asserted,enables the multiplexer 1460 to couple either the C(L) FLICK UP (3:0)LAT or SEL PMCL/ABSN signals as the SEL INP DATA (L) signals, and whennegated enables it to couple the C(R) FLICK UP (3:0) LAT or PMCR signalsas the SEL INP DATA (L) signals. Finally, a latch/buffer select circuit1473 provides a low-order SEL PARK BUF LAT (LFT) select left park bufferlatched signal that enable the multiplexer 1460 to select between thePMCR and the C(R) FLICK UP (3:0) LAT signals if the left/right selectcircuit 1472 is negating the SEL LEFT LAT signal, and between the C(L)FLICK UP (3:0) LAT and SEL PMCL/ABSN signals if the left right circuit1472 is asserting the SEL LEFT LAT signal.

The left/right select enable circuit 1467 provides two signals forcontrolling the various circuits 1471 through 1473 of the multiplexercontrol section 1466 in unison, namely, a low-order select enableportion 1474 and a high-order select enable portion 1475. The low-orderselect enable portion 1474 provides an asserted RCV ST 0 (NOTNPAC)/10/11/MS SCB signal generally during three timing periods asidentified by the asserted UP RCV ST (12:0) signals. In particular, theRCV ST 0 (NOT NPAC)/10/11/MS SCB signal is asserted if (a) the RCV ST 0timing signal is asserted, and if the CL/NPAC and CR/NPAC signals arenot both asserted, (b) the RCV ST 10 timing signal is asserted, and (c)the RCV ST 11 timing signal is asserted.

In addition, the signal is asserted if the control network messagepacket 60 being transmitted by the up output packet assembler 1422 is amultiple-source message enabling a scan backward operation, as indicatedby an asserted OUT MS SCB signal from the message packet prioritycircuit 1470. It will be appreciated that, if the RCV ST 0 timing signalis asserted, the first flick of the control network message packet 60 isbeing received, which contains the message type field 64. If the CL/NPACor CR/NPAC signals are then asserted, the control network message packet60 received from the respective left or right child is of the NPAC nilpacket type.

Similarly, if the RCV ST 10 or RCV ST 11 signal is asserted, flicks tenand eleven of the control network message packet 60 are currently beingreceived. However, the multiplexer control section 1466 provides a delayof one tick of the NODE CLK signal in controlling the multiplexer 1460,and so the control of multiplexer 1460 enabled by the RCV ST 0(NOTNPAC)/10/11/MS SCB signal will be effective with flicks eleven andtwelve, that is, the last two flicks of the control network messagepacket 60. It will be appreciated that these flicks contain the globalinformation field 71 and the checksum field 63.

The high-order select enable portion provides a RCV ST 1-9 signal whichis asserted when the UP RCV ST 1 through UP RCV ST 9 signals areasserted, and is otherwise negated. As noted above, the multiplexercontrol section 1466 provides a delay on one tick of the NODE CLK signalin controlling the multiplexer 1460, and so the control of themultiplexer 1460 enabled by the RCV ST 1-9 signal will be effective withflicks two through ten, which contain the combine function field 66 andthe data nibbles 70(i) (FIG. 5).

It will be appreciated that the RCV ST 0(NOT NPAC)/10/11/MS SCB and RCVST 1-9 signals are not asserted contemporaneously, but insteadcontemporaneous with receipt of different portions of a control networkmessage packet 60. In one embodiment, the RCV ST 0(NOT NPAC)/10/11/MSSCB signal will be asserted during receipt of the message type field 64,unless the message type is NPAC nil packet, and the

Each of the enable and select circuits 1471 through 1473 includes amultiplexer 1476 through 1478 and a flip-flop 1480 through 1482 dockedby the NODE CLK signal. The output signals provided by the flip-flops1480 and 1482 comprise the enable and select signals for the multiplexer1460, and so the flip-flops 1480 through 1482 effectively provide theone-tick delay noted above. The multiplexers 1477 through 1478 arecontrolled in unison by the RCV ST 0(NOT NPAC)/10/11/MS SCB and RCV ST1-9 signals. Thus, while the RCV ST 0(NOT NPAC)/10/11/MS SCB signal isasserted, the multiplexer 1476 in enable circuit 1472 couples a CLACTIVE left child active signal as a SEL RCV DATA OUT EN (L) selectreceive data out enable (left) signal to the data input terminal offlip-flop 1480, which is latched at the next tick of the NODE CLKsignal. The CL ACTIVE signal is controlled by a control register (notshown) on control network node 51, which is set by the diagnosticnetwork 16 if the node is connected to a left child. In that case, themultiplexer 1460 is enabled to provide SEL INP DATA (L) selected leftinput data signals.

Contemporaneously, the left/right select circuit 1472 couples anasserted signal (as identified by "+" on the FIG.) as a SEL LEFT selectleft signal to the data input terminal of flip-flop 1481. The flip-flop1481 is set at the next tick of the NODE CLK signal, to provide anasserted SEL LEFT LAT select left latch signal. The asserted SEL LEFTLAT signal-enables the multiplexer 1460 to couple either the C(L) FLICKUP (3:0) LAT signals or the SEL PMCL/ABSN signals as the SEL INP DATA(L) signals.

In one embodiment, in the right child receiver/buffer 1420(R), themultiplexer corresponding to multiplexer 1477 receives a negated signalinstead of the as serted signal. In that case, the corresponding SEL RTselected right output signal is negated, resulting in a negated SEL RTLAT selected right latched signal.

Finally, the latch/buffer select circuit 1473 couples a PKT PRIORITY (L)packet priority signal as a SEL PARK BUF (LFT) select park buffer leftsignal to the data input terminal of flip-flop 1482. The PKT PRIORITY(L) signal is controlled by the message packet priority circuit 1470,and is generally asserted if the left child receiver/buffer 1420(L) isreceiving an idle message packet, and the right child receiver/buffer1420(R) is receiving either a idle, abstain, NPAC nil packet or amultiple-source message packet. If the right child receiver/buffer1420(R) is receiving such a control network message packet 60, the PKTPRIORITY (L) signal is asserted, to enable the latch/buffer selectcircuit 1473 to, in turn, enable the multiplexer 1460 to couple signalsfrom the park buffer 1442 as the SEL INP DATA (L) signals. Otherwise,the multiplexer 1460 is conditioned to couple the C(x) FLICK UP (3:0)LAT signals as the SEL INP DATA (L) signal. As noted above, while theRCV ST 0(NOT NPAC)/10/11/MS SCB is asserted, the multiplexer 1478couples the PKT PRIORITY (L) as the SEL PARK BUF (LFT) signal, which islatched by flip-flop 1482 at the next tick of the NODE CLK signal. Theflip-flop 1482 provides the SEL PARK BUF LAT (LFT) signal.

While the RCV ST 1-9 signal is asserted, if the packet priority circuit1470 is negating the OUT MS SCB output multiple-source scan backwardsignal, the RCV ST 1-9 signal enables the multiplexers 1476 through 1478to couple the output signals of the respective flip-flops 1480 through1482 to their respective data input terminals. Thus, the flip-flopsmaintain their state at the successive ticks of the NODE CLK signal, andthe multiplexer 1460 continues to couple the selected SEL PMCL/ABSN orC(L) FLICK UP (3:0) LAT signals as the SEL INP DATA (L) signals.

However, if the OUT MS SCB signal is asserted, which may occur while theUP RCV ST 2 timing signal is asserted to indicate that a scan backwardoperation is taking place, the RCV ST 0(NOT NPAC)/10/11/MS SCB is alsoasserted. In that case, the multiplexers 1476 through 1478 couple thecorresponding signals the flip-flops in corresponding enable and selectcircuits in the right child receiver/buffer 1420(R) to the flip-flops1480 through 1482, where they are latched at the next tick of the NODECLK signal.

In that case, if the RCV DATA OUT EN (R) receive data out enable (right)signal is asserted, the right child is active, and so the flip-flop 1480will be set to assert the RCV DATA OUT EN (L) signal to enable theoutput of multiplexer 1460. In addition, the SEL RT LAT signal from theright child receiver/buffer 1420(R) will be negated, enabling theflip-flop 1481 to be cleared to negate the SEL LEFT LAT signal. Whenthat occurs, the multiplexer 1460 is enabled to couple the PMCR or C(R)FLICK UP (3:0) LAT signals as the SEL INP DATA (L) signals. The SEL PARKBUF LAT (RT) select park buffer latched (right) signal from right childreceiver/buffer 1420(R), latched by flip-flop 1482, will control whichof the PMCR or C(R) FLICK UP (3:0) LAT signals will be coupled by themultiplexer 1460.

When the UP RCV ST 2 signal is again negated, the OUT MS SCB signal isalso negated, which negates the RCV ST 0(NOT NPAC)/10/11/MS SCB. Thus,for the rest of the time the RCV ST 1-9 is asserted, the multiplexers1476 through 1478 will couple the output signals from the flip-flops1480 through 1482 to their data input terminals, enabling, in turn, themto maintain the condition they were in While the UP RCV ST 2 signal wasasserted. Thus, if the OUT MS SCB signal was negated during theassertion of the UP RCV ST 2 signal, the multiplexer 1460 will coupleeither the SEL PMCL/ABSN or the C(L) FLICK UP (3:0) LAT signals,originating from the left child node, as the SEL INP DATA (L) signals.

On the other hand, if the OUT MS SCB signal was asserted at that point,indicating a scan backward operation, the multiplexer 1460 will coupleeither the PMCR or C(R) UP (3:0) LAT signals, originating from the rightchild node, as the SEL INP DATA (L) signals. Thus, the enable and selectcircuits 1471 through 1473, and in particular the left/right selectcircuit 1472, enables the left child buffer/receiver 1420(L) to couplesignals from the right child node as the left input signals if a scanbackward operation is occurring. It will be appreciated that, while thereversal is initiated by the OUT MS SCB signal in synchronism with theUP RCV ST 2 signal, because of the one-tick delay provided by the enableand select circuits 1471 through 1473, the SEL INP DATA (L) signalsreflect the reversal in flick three, which is the beginning of the datanibbles 70(i). In addition, the reversal will end in response to thenegation of the RCV ST 1-9 signal, and, because of the one-tick delay,will be after flick 10, which contains the last of the data nibbles70(i).

The tag input selector 1445 also includes a multiplexer 1490 and enableand select circuits 1491 through 1493. The multiplexer 1490 selects P OV"x" parked overflow signals ("x" reference left or right) and C(x) TAGsignals, which corresponds to the C(x) FLICK UP (4) LAT signals, as theSEL INP TAG (L) selected input tag signals. The multiplexer 1490 iscontrolled by enable and select signals in a manner similar that ofmultiplexer 1460. In addition, the enable and select circuits 1491through 1493 are controlled by the RCV ST 0(NOT NPAC)/10/11/MS SCB andRCV ST 1-9 signals in a manner similar to that of the enable and selectcircuits 1471 through 1473.

FIGS. 12B-1C through 12B-1E depict details of the park buffer 1442,multiplexer 1452 and the park buffer control/status circuit 1450.Generally, FIGS. 12B-1C and 12B-1D include circuitry for controllingstorage of data from a control network message packet 60 in the parkbuffer 1442. FIG. 12B-1E includes circuitry for indicating the status ofthe park buffer, that is, whether it is storing data from a controlnetwork message packet 60 and, if so, whether the packet's segment bit77 (FIG. 5) was set.

With reference to FIGS. 12B-1C and 12B-1D, the multiplexer 1252 iscontrolled by a multiplexer control circuit 1500, which generates the PKBUF SRC SEL., (L) packet buffer source select (left) signal. If the PKBUF SRC SEL (L) signal is asserted, multiplexer 1452 couples the SEL INPDATA (L) signals to data input terminals of the park buffer 1442 forstorage. The park buffer 1442 buffers the signals at its data inputterminals at successive ticks of the NODE CLK signal. On the other hand,if the PK BUF SRC SEL (L) signal is negated, the multiplexer 1452couples the PMCL signals at the park buffer's output terminals to itsinput terminals.

The multiplexer control circuit 1500 asserts the PK BUF SRC SEL (L)signal under several circumstances. In particular, if the up outputpacket assembler 1422 is asserting the OUT PK MS signal, indicating thatit is transmitting a multiple-source message packet, several of gates1501 through 1504 will be energized if the multiplexer control section1466 (FIG. 12B-1A) is asserting both the SEL LEFT LAT and SEL PARK BUFLAT (LFT) signals, or the corresponding circuit in the right childreceiver/buffer 1420(R) is asserting both the SEL RT LAT and SEL PARKBUF LAT (RT) signals, and if the up control circuit 1424 is assertingthe UP RCV ST 1 through UP RCV ST 10 timing signals. This occurs if thesource data selector 1440 in either child receiver buffer 1420(x) iscoupling signals from its respective park buffer as its SEL INP DATA (x)selected input data signals.

In addition, the multiplexer control circuit 1500 asserts the PK BUF SRCSEL (L) park buffer source select signal if the up output packetassembler 1422 is negating the OUT PKT MS output packet multiple sourcesignal, indicating that it is transmitting a message packet of otherthan the multiple source message type. In that case, if a PARK PKT (L)park packet (left) signal is asserted, gates 1505 and 1506 enable the PKBUF SRC SEL (L) to be asserted while the up control circuit 1424 isasserting the UP RCV ST 1 through UP RCV ST 10 timing signals. The PARKPKT (L) signal is generated by circuitry depicted on FIG. 12B-1D.

With reference to FIG. 12B-1D, a park enable circuit 1510 generates aPARK EN (L) park enable (left) signal, which, when asserted, enables acontrol network message packet 60 being received by the left childreceiver buffer 1420(L) to be buffered in park buffer 1442. The parkenable circuit 1510 asserts the PARK EN (L) signal in response toconditions, each represented by an AND gate 1511 through 1515. If anyone of the conditions is satisfied, when the UP RCV 0 receive statesignal is asserted, the park enable circuit asserts the PARK EN (L)signal.

In particular, AND gate 1511 asserts a CLMS/SRSS signal if the leftchild node is providing a multiple-source message, and the right childnode is providing a single-source message packet. As noted above, thesingle-source message has priority, and so the left childbuffer/receiver 1420(L) will buffer the multiple-source message packet.If the left child node provides a multiple-source message, the CL/MSleft child multiple-source signal is asserted when the UP RCV ST 0signal is asserted, and when the right child node is providing asingle-source message the CR/SS right child single-source signal isasserted, when the UP RCV ST 0 signal is asserted.

Similarly, AND gate 1512 asserts a CLMS/CR IDLE/NO (R) PARKED signal ifboth (a) the left child node is providing a multiple-source messagepacket, and (b) the right child node is providing an idle message, asindicated by the asserted CR/IDLE signal and the park buffer in theright child receiver/buffer 1420(R) is not buffering a message packet,as indicated by a negated PARKED (R) signal. In that case, the rightchild receiver/buffer 1420(R) has no message packet to be used inconnection with the multiple-source message packet being received by theleft child receiver/buffer, and so the PARK EN (L) signal is asserted toenable the multiple-source message packet to be buffered.

The remaining gates 1513 through 1515 of the park enable circuit 1500enable assertion of the PARK EN under several conditions while theparent node is disabling flow, as indicated by the negation of an LIPFLOW OK signal. The LIP FLOW OK signal is derived from the FLOW CTRL(UP) flow control signals received from the flick down control portion1400. While the UP FLOW OK signal is asserted, the flick up controlportion 1400 can transmit control network message packets 60 to itsparent, but when negated the flick up control portion is disabled fromtransmitting multiple-source message packets thereto. In particular, ifthe UP FLOW OK signal is negated, and if the right childreceiver/buffer's park-buffer is buffering a message packet 60, whilethe left child receiver/buffer 1420(L) is receiving a multiple-sourcemessage, a CLMS/(R) PARKED/FLOW OFF signal is asserted, enablingassertion of the PARK EN (L) signal. Similarly, if the left childreceiver/buffer 1420(L) is receiving a multiple-source message, whilethe right child node is providing an absolute or a NPAC nil packetmessage, AND gate 1514 asserts the CLMS/(R) ABSNPAC/FLOW OFF signal,which enables assertion of the PARK EN (L) signal. Finally, if bothchild nodes are providing multiple source messages, AND gate 1515asserts a (L)(R) MS/FLOW OFF signal, which also enables assertion of thePARK EN (L) signal.

A park control circuit 1516, also shown on FIG. 12B-1D, generates thePARK PKT (L) park packet signal which controls gates 1505 and 1506 (FIG.12B-1C). The park control circuit 1516 also generates signals whichindicate whether the segment bit 77 (FIG. 5) of the control networkmessage packet 60 being buffered in the park buffer 1442 is set orcleared. If the segment bit is cleared, a multiplexer 1517, undercontrol if the PARK EN (L) signal and the negated SEL INP TAG (L)signal, which at that point corresponds to the segment bit 77, enables aflip-flop 1518 to be set to assert a PARK/NO SEG park/segment clearsignal. The asserted PARK/NO SEG signal indicates that a control networkmessage packet 60 whose segment bit is cleared is being buffered in parkbuffer 1442. On the other hand, if the segment bit is set, a multiplexer1520, under control of the asserted PARK EN (L) signal and asserted SELINP TAG (L) signal, enables a flip-flop 1521 to be set to assert aPARK/SEG park/segment set signal. The PARK/SEG and PARK/NO SET signalsare both coupled to an OR gate 1522, which generates the PARK PKT (L)signal which is coupled to AND gates 1505 and 1506 (FIG. 12B-1C).

FIG. 12B-1E depicts park buffer status circuitry 1530 which indicatesthe status of the park buffer 1442 (FIG. 12B-1A). Normally, the parkbuffer status circuitry 1530 will indicate, after a control networkmessage packet 60 has been buffered in the park buffer 1442, that apacket has been buffered, and whether the packet's segment bit 77 wasset. However, under some circumstances, if, for example, a set flush bit75 is received in the either control network message packet 60 beingbuffered in a control network message packet 60 being received by theright child receiver/buffer 1420(R), the control network message packet60 buffered is flushed, in which case park buffer status circuitry 1530is conditioned to indicate that the packet buffer 1442 is empty.Further, if the source data selector 1440 couples the contents of thepark buffer 1442 as the SEL INP DATA (L) signals, the park buffer statuscircuitry 1530 is conditioned to indicate that the packet buffer isempty.

More specifically, the park buffer status circuitry 1530 includes aparked/segment clear status section 1531 and a parked/segment set statussection 1532, and a clear control section 1533. The parked/segment clearstatus section 1531 generates a PARKED/NO SEG parked/segment clearsignal that, when asserted, indicates that a control network messagepacket 60 whose segment bit 77 was clear is parked park buffer 1442. Theparked/segment set status section 1532, on the other hand, generates aPARKED/SEG parked/segment clear signal that, when asserted, indicatesthat a control network message packet 60 whose segment bit 77 was set isparked park buffer 1442. The PARKED/NO SEG and PARKED/SEG signals, whenasserted, energize an OR gate 1533 which generates a PARKED (L) parkedsignal which controls a gate corresponding to AND gate 1512 in the rightchild receiver/buffer 1420(R) to control the operations of the parkenable circuitry therein.

The clear control section 1533 controls a CLR PK BUF STATUS clear parkbuffer status signal. Section 1533 includes a flush control portion 1537that asserts a FLUSH/ST 10 signal in response to the coincidence of theFLUSH LAT signal and the UP RCV ST 10 signal. The SEL INP TAG (L)selected input tag signal from tag input selector 1445 and SEL INP TAG(R) from the right child receiver/buffer 1420(R) are coupled to an ORgate 1540, which provides a COMP TAG composite tag signal to one datainput terminal of a multiplexer 1541. If the LIP RCV ST 9 signal is thenasserted, the COMP TAG signal represents the flush bit 75 of the controlnetwork message packet 60 being received, and so at this point themultiplexer 1541 couples COMP TAG signal is coupled as a FLUSH signal toa flip-flop 1542. The flip-flop 1542 latches the FLUSH signal at thenext tick of the NODE CLK signal to provide the FLUSH LAT signal. If theFLUSH LAT signal is negated, neither control network message packet 60being received by a child buffer/receiver 1420(x) has a set flush bit75. On the other hand, if the FLUSH LAT signal is asserted, at least onesuch control network message packet 60 does have a set flush bit 75.

Clear control section 1533 also has a circuit 1543 that generates a USEPK BUF signal if the input source identifier enables the source dataselector 1440 (FIG. 12B-1) to use the control network message packet 60buffered in the park buffer 1442. This is indicated if both the SEL LEFTLAT and SEL PARK BUF LAT (LFT) signals are asserted by left/right selectcircuit 1472 and latch/buffer select circuit 1473 in synchronism withthe RCV ST 0 signal. In that case, the USE PK BUF signal is asserted.

The parked/segment clear status section 1531 includes a multiplexer 1534and flip-flop 1535. Multiplexer 1534 is controlled by an AND gate 1536,which is energized if the UP RCV ST 10 and PARK/NO SEG signals areasserted, and if the FLUSH LAT signal is negated. As noted above, theFLUSH LAT signal is controlled by the clear control section 1533, and isnegated if no control network message packet 60 is received in which theflush bit 75 is asserted. If the clear control section is negating a CLRPK BUF STATUS clear park buffer status signal, multiplexer 1534 couplesan asserted signal to the flip-flop 1535, which is set in response tothe next tick of the NODE CLK signal to assert the PARKED/NO SEG signal.On the other hand, if the FLUSH LAT signal is asserted, the AND gate1536 is de-energized, but the clear control section 1533 will beasserting the CLR PK BUF STATUS signal. In that case, multiplexer 1534will couple a negated signal to flip-flop 1535, to reset the flip-flop1535 and negate the PARKED/NO SEG signal.

After parked/segment clear status section 1531 has established thecondition of the PARKED/NO SEG signal while a control network messagepacket 60 is loaded into the park buffer 1442, the multiplexer 1534normally couples the PARKED/NO SEG signal to the data input terminal ofthe flip-flop 1535 to enable it to maintain its condition. However, theclear control section may enable a change in the PARKED/NO SEG signalunder two circumstances, namely, if a control network message packet 60is received in which the flush bit 75 is set, or if the buffered packetis used by the source data selector 1440. If, while the packet is beingbuffered, a control network message packet 60 is received in which theflush bit 75 is set, the FLUSH/ST 10 signal will be asserted, enablingassertion of the CLR PK BUF STATUS clear park buffer status signal.Similarly, if the source data selector 1440 selects the buffered packetas the SEL INP DATA (L) signal, the USE PK BUF signal is asserted, alsoenabling assertion of the CLR PK BUF STATUS signal. In either case, theasserted CLR PK BUF STATUS signal enables the multiplexer 1534 tocoupled a negated signal to the data input terminal of the flip-flop1534, enabling the flip-flop 1534 to be cleared to negate the PARED/NOSEG signal.

The parked/segment set status section 1532 operates in a manner similarto the parked/segment clear status section 1531 if the PARK/SEG signalis asserted instead of the PARK/NO SEG signal.

FIG. 12B-1F depicts details of the park overflow buffer 1446,multiplexer 1453 and park overflow buffer (left) status/control circuit1451. With reference to FIG. 12B-1F, the park overflow buffer 1446comprises a flip-flop, identified by the same reference numeral. Thestatus/control circuit 1451 includes a condition-circuit 1544 and aclear enable circuit 1545. The condition circuit 1544 enables theflip-flop 1446 to be conditioned in response to the scan overflow bit 80of the control network message packet 60. In particular, if a controlnetwork message packet 60 is being stored in the park buffer 1442 an ANDgate 1546 is energized in coincidence with the assertion of the UP RCVST signal to assert the PARK LEFT OVFL BUF park left overflow buffersignal. The asserted PARK LEFT OVFL BUF signal enables the multiplexer1453 to couple the SEL INP TAG (L) selected input tag signal from taginput selector 1445 (FIG. 12B-1) to the data input terminal of flip-flop1446. The signal conditions flip-flop 1446 at the next tick of the NODECLK signal.

Thereafter, the PARK LEFT OVFL BUF signal is negated. If a CRL LEFT OVFLBUF clear left overflow buffer signal from clear enable circuit 1545 isalso negated, multiplexer 1453 couples the POVL park overflow leftsignal from flip-flop 1446 back to its input terminal, to enableflip-flop 1446 to maintain its condition at successive ticks of the NODECLK signal. If the clear enable circuit 1545 thereafter asserts the CLRLEFT OVFL BUF signal, the multiplexer 1453 couples a negated signal tothe data input terminal of the flip-flop 1446, enabling the flip-flop tobe cleared at the next tick of the NODE CLK signal, to negate the POVLsignal.

The clear enable circuit 1545 asserts the CLR LEFT OVFL BUF signal underfour circumstances. First, the clear enable circuit 1545 asserts the CLRLEFT OVFL BUF signal if the up control circuit 1424 asserts the UP RCVST RST signal, which occurs if both child nodes are root nodes. Second,the clear enable circuit 1545 asserts the signal if an AND gate 1547 isenergized, which occurs if flush control portion 1537 (FIG. 12B-1E)asserts the FLUSH LAT signal indicating reception of a control networkmessage packet 60 in which the flush bit 75 is set. In addition, if ANDgates 1550 or 1551 are energized, indicating respectively, that a packetbuffered in packet buffer in the left or right child receiver/buffer1420(x), is coupled by the respective source data selector 1440 as theSEL INP DATA (x) signals.

Finally, FIG. 12B-1G depicts a segment bit latch 1560 maintained by thepark buffer control/status circuit 1450. the segment bit latch 1560provides a SEG L segment left signal that indicates the condition of thesegment bit 77 in a control network message packet 60 received from theleft child if it is a multiple-source message enabling a scan forward orreduce operation, or of a packet received from the right child it is amultiple-source message enabling a scan backward operation. The SEG Lsignal indicates the state of the segment bit of the control networkmessage packet 60 regardless of whether the multiple-source message datais processed when received, or if it is buffered in the packet buffer1442.

The segment bit latch 1560 includes a flip-flop 1561, which generatesthe SEG L signal, and a multiplexer 1562 which controls the source ofsignals coupled to the data input terminal of the flip-flop 1561. Asegment latch source control circuit 1563 generates a SEG BIT SRCE SELsegment bit source select signal to control the source of the signalcoupled by the multiplexer 1562 to the flip-flop 1561, effectivelyselecting between the C(L) FLICK UP (4) LAT signal, representing thereceived tag signal, or the PARKED/SEG parked/segment signal fromparked/segment set status section 1532 (FIG. 12B-1E). A segment latchhold select circuit 1564 generates a SEG BIT HOLD segment bit holdsignal to enable the multiplexer 1562 to maintain its condition, or tointerchange conditions with the segment bit latch in the right childreceiver/buffer 1420(R). The segment latch hold select circuit 1564asserts the SEG BIT HOLD signal contemporaneous with the assertion bythe up control circuit 1424 of the UP RCV ST 1 through UP RCV ST 9signals.

If neither the SEG BIT SRCE SEL signal nor the SEG BIT HOLD signal isasserted, the multiplexer 1563 couples the C(L) FLICK UP (4) LAT signalas a SEL SEG (L) selected segment (left) signal to the data inputterminal of flip-flop 1561. The flip-flop 1561 latches the SEL SEG (L)signal at each tick of the NODE CLK signal. The circuit comprisingsegment latch source control 1563 is similar to the circuit of themessage packet priority circuit 1470, and provides an asserted SEG BITSRCE SEL signal, contemporaneous with the assertion of the UP RCV ST 0signal, if the control network message packet 60 being received is to bebuffered in the park buffer 1442. In that case, the asserted SEG BITSRCE SEL signal enables the multiplexer 1562 to couple the PARKED/SEGparked/segment signal from parked/segment set status section 1532 (FIG.12B-1E), which indicates whether the segment bit 77 of the bufferedcontrol network message packet 60 was set.

When the UP RCV ST 0 signal is negated, the SEG BIT SRCE SEL signal isnegated, and the segment latch hold select circuit 1564 asserts the SEGBIT HOLD signal. This enables the multiplexer 1562 to couple the SEG Lsignal from flip-flop 1561 as the SEL SEG (L) signal to the data inputterminal of flip-flop 1561, enabling it to maintain its state while theSEG BIT HOLD signal is asserted. If, while the SEG BIT HOLD signal isasserted, an AND gate 1565 in the segment latch source control circuit1563 is also energized, contemporaneous with the assertion of the UP RCVST 2 signal, to assert the SEG BIT LAT SRCE SEL signal. This will occurif the control network node 51 is transmitting a multiple-source messageenabling a scan backward operation. In that case, the multiplexer 1562couples a SEG R segment right signal from a segment bit latch 1560 inthe right child receiver/buffer 1420(R) as the SEL SEG (L) selectedsegment (left) signal, which the flip-flop 1561 latches. In that case,the conditions of the SEG "x" ("x" indicates "L" or "R") are effectivelyinterchanged, thereby completing the interchange of data from controlnetwork message packets 60 from the left and right child nodes. When theUP RCV ST 2 signal is then negated, the SEG BIT SRCE SEL signal is againnegated, to enable only the SEG BIT HOLD signal to control themultiplexer 1562.

When the SEG BIT HOLD signal is later negated, contemporaneous with theassertion of the UP RCV ST 10 signal, the SEG BIT HOLD signal isnegated, enabling the multiplexer 1562 to couple the C(L) FLICK UP (4)LAT signal to the tip-flop 1561 as the SEL SEG (L) signal.

ii. Flick (Up) Data Processor 1421

FIG. 12B-2 depicts details of the flick (up) data processor 1421. Withreference to FIG. 12B-2, the flick (up) data processor 1421 includes theadder 1425, OR circuit 1426, XOR (exclusive OR) circuit 1427 andcomparator 1428, as represented on FIG. 12B. The adder 1425 provides, inresponse to the SEL INP DATA (x) selected input data signals from theleft and right child receiver buffers 1420(x), SUM signals an UP CRY OUTcarry signal and an OVFL overflow in signal, which together representthe sum of the values represented by the SEL INP DATA (x) signals.Associated with the adder 1425 is a carry latch 1570 and a carry selectmultiplexer 1571 that, under control of a control circuit 1577,selectively latches the UP CRY OUT carry signal. The carry latch 1570,at each tick of the NODE CLK signal provides a CRY IN carry in signalwhich is also used by the adder 1425 in generating the SUM signals.Similar circuitry (not shown) is provided for the OVFL overflow signal,to provide the OVFL IN signal. In addition, if the SEL INP DATA (x)signals represent the last data nibbles 70(i) of their respectivecontrol network message packets 60, and if the control network messagepackets 60 enable a scan or reduce operation, the up output packetassember 1422 uses the CRY IN and OVFL IN signal in conditioning thescan overflow bit 80 of the control network message packet 60 it istransmitting.

The OR circuit 1426 and XOR circuit 1427 generate OR and XOR signalsrepresenting the bit-wise OR and XOR, respectively, of the SEL INP DATA(x) signals. In addition, the complements of the XOR signals aredirected to an AND gate 1574. If all of the XOR signals are negated,which occurs if the corresponding bits of the SEL INP DATA (L) and SELINP DATA (R) signals have the same value, the AND gate 1574 is energizedto assert a L EQ RT left equals right signal. Associated with the XORgate 1427 is a latch 1575 and a multiplexer 1576 that selectivelylatches the L EQ RT signal. The latch 1575, at each tick of the NODE CLKsig nal, generates a DATA EQ LAT data equals latched signal.

The comparator 1428 generates L GT RT left greater than right signalwhose condition indicates whether the binary-encoded value of the SELINP DATA (L) signals is greater than the binary-encoded value of the SELINP DATA (R) signals. Associated with the comparator 1428 is a latch1572 and multiplexer 1573 which selectively latches the L GT RT signal.The multiplexer 1573, along with multiplexer 1576, are controlled by thecontrol circuit 1577. Generally, while the DATA EQ LAT signal isasserted, indicating that the binary-encoded value of the successivedata nibbles 70(i), represented by the SEL INP DATA (x) signals atsuccessive ticks of the NODE CLK signal, are equal, the control circuit1577 enables the multiplexers 1576 and 1573 to direct the L EQ RT and LGT RT signals, respectively, to their respective latches 1575 and 1572.

However, if the L EQ RT signal is negated, indicating that thebinary-encoded values of the SEL INP DATA (x) signals are not equal, thelatch 1575 negates the DATA EQ LAT signal. At that point the L GT RTsignal, which is latched by the latch 1572, indicates which of the SELINP DATA (x) signals has the greater binary-encoded value. The controlcircuit 1577 thereafter enables the multiplexers 1576 and 1573 to couplethe output signals from their latches 1572 to their input terminals, sothat they maintain their conditions at succeeding ticks of the NODE CLKsignals. Thus, the control circuit 1577 ensures that the COMP L/R LATsignal will identify the control network message packet 60, representedby the SEL INP DATA (x) signals, having a packet data portion 62 withthe greater binary-encoded value.

The control circuit 1577 generates two control signals for controllingthe multiplexers 1573 and 1576, namely, a LAT EQ/COMP LAT latchequal/comparator latch signal and a HOLD EQ/COMP LAT holdequal/comparator latch signal. While the LAT EQ/COMP LAT signal isasserted, the multiplexers 1573 and 1576 couple the L GT RT and L EQ RTsignal to the data input terminals of their respective latches 1572 and1575. Initially, contemporaneous with the assertions of the UP RCV ST 0through LIP RCV ST 2 signals by the up control circuit 1424, both theHOLD EQ/COMP LAT and LAT EQ/COMP LAT signals will be negated. In thatcondition, the multiplexers 1573 and 1573 couple asserted signals to thedata input latches of their respective latches 1572 and 1575, to set thelatches and assert the COMP L/R LAT and DATA EQ LAT signals.

However, while the UP RCV ST 3 through UP RCV ST 9 signals are asserted,which occurs while the SEL INP DATA (x) signals represent the successivedata nibbles 70(i) of the control network message packets 60, an ANDgate 1580 asserts the LAT EQ/COMP LAT signal, if the latch 1575 isasserting the DATA EQ LAT signal, indicating that the binary-encodedvalues of the data nibbles 70(i) are equal, and if the up output packetassember 1422 is asserting either the UP OUT PKT MS or UP OUT PKT SSsignals. The last condition indicates that the data represented by atleast one of the SEL INP DATA (x) signals is from a single-source ormultiple-source message. If the DATA EQ LAT signal is negated,indicating that the binary-encoded values of the SEL INP DATA (x)signals are not the same, the LAT EQ/COMP LAT signal is negated.

The control circuit 1577 asserts the HOLD EQ/COMP LAT signal underseveral circumstances. If, while the UP RCV ST 3 through UP RCV ST 9signals are asserted, an UP BOTH SS signal is asserted, and if the DATAEQ LAT signal is negated, an AND gate 1581 is energized to enable an ORgate 1582 to assert the HOLD EQ/COMP LAT signal. The UP BOTH SS signalis asserted if both child nodes are transmitting single-source messages.In this case, if both single-source message are of the configurationtype, and if AND gate 1581 is energized, the data nibbles 70(i)single-source messages will represent different height values. As notedabove, if control network node 51 receives single-source messages, ofthe configuration type, having different height values, it transmits asingle-source message in which the height value is the maximum of thetwo received height values. AND gate 1581 enables this to occur.

In addition, if, while the UP RCV ST 3 through LIP RCV ST 9 signals areasserted, if the up output packet assembler 1422 is asserting the UP OUTPKT MS up output packet multiple-source signal, indicating that it istransmitting a multiple-source message, an AND gate 1583 is energized toenable the OR gate 1582 to assert the HOLD EQ/COMP LAT signal. Thisenables the latch 1572 to maintain its condition at that point, and, ifthe multiple-source message enables a maximum operation, the up outputmessage assembler may use the COMP L/R LAT signal in determining whichof the SEL INP DATA (x) signals to use in the control network messagepacket 60 it is transmitting.

In addition, if the control network message packet 60 being transmittedby the up output packet assember 1422 is multiple-source messageenabling a multiple-word operation, which effectively uses subsequentcontrol network message packets 60 of the multiple-source message type,an AND gate 1584 enables the OR gate 1582 to maintain the HOLD EQ/COMPLAT signal asserted. That a multiple-source message enables a multipleword operation is indicated by the assertion, by the up output packetassembler 1422 of a UP M SCAN multiple-word scan signal, along with theassertion of the UP OUT PKT MS signal. When the up control circuit 1421then asserts the UP RCV ST 12, AND gate 1584 is energized to enable theOR gate 1582 to assert the HOLD EQ/COMP LAT signal.

It will be appreciated that the adder 1425, OR circuit 1426, XOR circuit1427 and comparator 1428 operate in response to each set of the SEL INPDATA (x) signals provided thereto, without regard to the types ofmessages and message packets as represented by the SEL INP DATA (x)signals, or the operations enabled thereby. The up output packetassembler determines which, if any, of the output signals provided byeach of the circuits 1425 through 1428 to use in assembling controlnetwork message packet 60 for transmission thereby. Thus, the circuits1425 through 1428 do not require or use enabling signals based on thetype of control network message packet 60 being assembled in order tooperate.

iii. Up Output Packet Assembler 1422

FIG. 12B-3 depicts a detailed block diagram of the up output packetassembler. With reference to FIG. 12B-3, the up output packet assembler1422 comprises an output data selector 1590 that receives signals from anumber of sources and, under control of OUT DATA SEL output data sourceselect signals from an output source identifier circuit 1592, couplessignals from one source as the low-order P FLICK LIP (3:0) signals. Inaddition, an output tag selector circuit 1592 that, under control of OUTTAG SEL output tag select signals from the output source identifiercircuit 1591, selectively couples signals from a number of sources asthe P FLICK UP (4) signal. It will be appreciated that the low-order PFLICK UP (3:0) signals, at successive ticks of the NODE CLK signal,represents the packet information portion of successive flicks of acontrol network message packet 60. Similarly, the high-order P FLICK UP(4) signal, also at successive ticks of the NODE CLK signal, representsthe successive tag signals in the control network message packet 60.

The up output packet assembler 1422 also includes an output packetstatus store 1593 that receives the P FLICK UP (3:0) signals, as well asthe UP RCV ST (2:0) signals and the NODE CLK signal and generates upoutput packet status signals identifying the message type, packet typeand, if the control network message packet 60 represented by the P FLICKUP (4:0) signals is of the multiple-source type, the particularoperations enabled thereby. In addition, the up output packet assembler1422 includes a checksum generator 1594 that, after being reset by theUP RCV ST 0 signal, at successive ticks of the NODE CLK signal providesCHECK (4:0) checksum signals in response to the P FUCK UP (4:0) signals.During transmission of the last flick of the control network messagepacket 60, containing the checksum field 63 (FIG. 5), the output sourceidentifier circuit 1591 enables the output data and tag selectors 1590and 1592 to couple the CHECK (3:0) and CHECK (4) checksum signals,respectively, as the P FLICK UP (3:0) and P FLICK UP (4) signals.

The structures of the various circuits depicted on FIG. 12B-4 will notbe described in detail. The output data and tag selectors 1590 and 1592each comprise one or more multiplexers, which, under control of the OUTDATA SEL and OUT TAG SEL signal, selectively couple signals inputthereto to their respective output terminals. The particular signalscoupled by the respective selectors 1590, as determined by the outputsource identifier circuit 1591, depends on several factors, includingwhich types of control network message packets 60 are being received bythe left and right child receiver/buffers 1420(x), on whether the parkbuffers 1442 in the left and right child receiver/buffers 1420(x) arestoring message packets, and on which UP RCV ST (12:0) state signal isbeing asserted. The output source identifier 1591 determines which typeof control network message packet 60 to generate, which information islatched in the output packet status store for use by the output sourceidentifier 1591 as it enables the output data selector 1590 to provide PFLICK UP (3:0) signals representing the successive flicks of the controlnetwork message packet 60.

In addition, the output source identifer 1591 in response to the INP PKTTYPE signals and the UP RCV ST (12:0) signals, enables the output tagselector 1592 to selectively couple signals input thereto as the P FLICKUP (4) signal. The particular input signal selected to be coupled willdepend on the particular UP RCV ST (12:0) signal asserted, so that thecontrol network message packet 60 generated has the correct sequence oftag bits represented by the successive P FLICK UP (4) signal.

Generally, the output data selector 1590 receives the SEL INP DATA (x)signals from the left and right child receiver/buffers 1420(x), as wellas the SUM, OR, and XOR signals from the flick (up) data processor 1421.In addition, the output data selector 1590 receives IDLE, MS and ABSNsignals representing the encoding of the message type field 64 for acontrol network message packet 60 of, respectively, the abstain,multiple-source and abstain message type. Finally, the output dataselector 1590 receives the CHECK (3:0) checksum signals from thechecksum generator 1594.

The output source identifier 1591 enables the output data selector 1590,at each tick of the NODE CLK signal, and in synchronism with the UP RCVST (12:0) signals, to couple one of these sets of signals to its outputterminals as the P FLICK UP (3:0) signals. In selecting which signals tobe coupled, the output source identifier uses INP PKT TYPE input packettype signals from the input packet type decoders 1447 in the childreceiver/buffers 1420(x), representing the types of messages receivedfrom the left and right children. Thus, if control network messagepackets 60 of different message types are being received from the childnodes, the output source identifier 1591 determines which, if either,type will be transmitted.

If, for example, the INP PKT TYPE signals indicate that a single-sourcemessage is being received from one child node and no message is beingreceived from the other child node, the output source identifier 1591will generate OUT DATA SEL signals to enable the output data selector tocouple the SEL INP DATA (x) signals from the child receiver/buffer1420(x) whose child node is providing the single-source message. Theoutput source identifier 1591 enables the output data selector 1590 tocouple the SEL INP DATA (x) signals while the UP RCV ST 0 through UP RCVST 10 signals are being asserted. In that case, the output packet statusstore 1593 asserts the LIP OUT PK SS up output packet single-sourcesignal while the UP RCV ST 0 is asserted in synchronism with the nexttick of the NODE CLK signal. In addition, if the single-source messageis of the configuration type, the output packet status store 1593asserts the UP OUT PKT CONFIG up output packet configuration signalwhile the UP RCV ST 1 is asserted in synchronism with the next tick ofthe NODE CLK signal. The output source identifier uses the UP OUT PKT SSand UP OUT PKT CONFIG signals to identify the type of packet it isgenerating as it generates OUT DATA SEL signals enabling generation ofsuccessive flicks of the message packet.

However, if INP PKT TYPE signals indicate that single-source messages ofthe configuration type are being received from both child nodes, theoutput source identifier 1591 will generate OUT DATA SEL signals toenable the output data selector to couple the SEL INP DATA (x) signalsfrom either child receiver/buffer 1420(x). If the COMP L/R LAT compareleft/right latched signal from comparator 1428 (FIG. 12A) indicates thatthe binary-encoded value of the packet data portion 62 from one child isgreater than the binary-encoded value of the packet data portion 62received from the other, it generates OUT DATA SEL signals to enable theoutput data selector at that point to begin coupling the SEL INP DATA(x) signals from that child's receiver/buffer 1420(x) as the P FLICK UP(3:0) signals. The output source identifier 1591-enables the output dataselector 1590 to couple the SEL INP DATA (x) signals while the UP RCV ST0 through UP RCV ST 10 signals are being asserted. In addition, theoutput packet status store 1593 asserts the UP OUT PK SS up outputpacket single-source signal and UP OUT PKT CONFIG up output packetconfiguration signal while, respectively, the UP RCV ST 0 and UP RCV ST1 signals are asserted in synchronism with the next tick of the NODE CLKsignal. The output source identifier uses the UP OUT PKT SS and UP OUTPKT CONFIG signals to identify the type of packet it is generating as itgenerates OUT DATA SEL signals enabling generation of successive flicksof the message packet.

However, if INP PKT TYPE signals indicate that multiple source messagesare being received from both child nodes, the output source identifier1591 will, in synchronism with the assertion of the UP RCV ST 0 signal,generate OUT DATA SEL signals to enable the output data selector 1590 tocouple the MS signals, which correspond to the MS message type code, asthe P FLICK UP (3:0) signals. The output packet status store 1593, inresponse, asserts the UP OUT PKT MS up output packet multiple-sourcesignal, which controls subsequent operations by the output sourceidentifier circuit 1591. The output source identifier circuit, at theassertion of the UP RCV ST 1 and UP RCV ST 2 signals, generates OUT DATASEL signals that enable the output data selector to couple the SEL INPDATA (x) signals from either of the child receiver/buffers 1420(x) asthe P FLICK UP (3:0) signals. The output data selector 1590 mayselectively couple either the SEL INP DATA (x) signals, since theoperation enabled by the multiple source messages will be the same.

The output packet status store 1593, enabled by the assertion of the UPRCV ST 1 signals, in synchronism with the tick of the NODE CLK signal,may also assert the OUT PKT SCF/RED output packet scan forward/reducesignal, an UP REDUCE signal and the UP M SCAN up multi-word scan signal.Thes signals, which are derived from the encoding of the packet typefields 65 and pattern bits 67 of the message packets 60 being received,provide information as to the operation enabled by the multiple-sourcemessages. If the OUT PKT SCF/RED signal is asserted, the operationenabled by the multiple-source message is either a scan forwardoperation or a reduce operation; if the signal is negated the operationis a scan backward operation. If the operation is a reduce operation,the output packet status store 1593 also asserts the UP REDUCE signals.In addition, if the operation is a multi-word scan operation, the outputpacket store 1593 asserts the UP M SCAN signal. The output sourceidentifier 1591 uses these signals in subsequent operations.

When the LIP RCV ST 2 signal is asserted, in synchronism with the tickof the NODE CLK signal, the output packet status store 1593 asserts oneof the UP COMB ADD, UP COMB OR, UP COMB XOR and UP COMB MAX up combineadd, OR, XOR, and maximum signals. These signals, which are derived fromthe encoding of the combine function field 66 of the message packets 60being received, provide additional information as to the operationenabled thereby. The output source identifier 1591 uses these signals insubsequent operations. In particular, if the UP COMB ADD, UP COMB OR, UPCOMB XOR signals are asserted at the assertions of the UP RCV ST 3through LIP RCV ST 10 signals, the output source identifier circuit 1591generates OUT DATA SEL signals to enable the output data selector tocouple the SUM, OR or XOR signals, respectively, as the P FLICK UP (3:0)signals.

On the other hand, if the UP COMB MAX signal is asserted, the outputsource identifer 1591 uses the COMP L/R LAT compare left/right latchedsignal 16 select one of the SEL INP DATA (L) or SEL INP DATA (R) signalsto be coupled as the P FLICK UP (3:0) signal, and generates appropriateOUT DATA SEL signals. In particular, if the COMP L/R LAT compareleft/right latched signal from comparator 1428 (FIG. 12A) indicates thatthe binary-encoded value of the packet data portion 62 from one child isgreater than the binary-encoded value of the packet data portion 62received from the other, it generates OUT DATA SEL signals to enable theoutput data selector at that point to begin coupling the SEL INP DATA(x) signals from that child's receiver/buffer 1420(x) as the P FLICK UP(3:0) signals.

If INP PKT TYPE signals indicate that a multiple-source message is beingreceived from one child node, and an idle message is being received fromthe other child node, the output source identifier uses the PK BUF (x)ST park buffer status signals to determine whether a message packet fromthe other child node is being buffered in a packet buffer. If so, theoperations described above are performed, in the same manner as ifmultiple-source messages were being received from both child nodes.Similarly, if the input packet type signals indicate that idle or NPACnil packet messages are being received from both child nodes, and the PKBUF (x) ST status signals indicate that park buffers 1442 in both childreceiver/buffers 1420(x) are buffering message packets, the operationdescribed above are performed, in the same manner if multiple-sourcemessages were being received from both child nodes.

In any case, while the UP RCV ST 11 signal is asserted, which occursduring transmission of the global information portion 71, the outputsource identifier generates OUT DATA SEL signals that enable the outputdata selector to couple the OR signals as the P FLICK UP (3:0) signals.This occurs regardless of the message types of the message packets 60received from the respective child nodes, or the message type of themessage packet 60 transmitted to the parent node. As noted above, theglobal information portion 71 in the packets 60 as transmitted by theleaves 21 (FIGS. 1, 4A and 4B) contains status information that is ORedby the control network 14 and the result broadcasted to all leaves inthe partition, including the scalar processors 12. Thus, by monitoringthe global information portion 71 of packets 60 that they receive, thescalar processors 12 can determine status of, for example, theprocessing elements 11.

In addition, while the UP RCV ST 12 signal is asserted, it enables theoutput data selector 1590 to couple the CHECK (3:0) checksum signals asthe P FLICK UP (3:0) signals to provide the low-order portion of thechecksum field 63 of the control network message packet 60 beinggenerated.

3. Root Flag 1407 And Associated Control Circuitry

FIG. 12C depicts details of the root flag 1407 (FIG. 12A) and circuitryfor controlling its condition. With reference to FIG. 12C, the controlcircuitry includes a left height comparison circuit 1600 and acorrespoding right height comparison circuit (not shown). Generally,each of the height comparison circuits, if the respective left or rightchild nodes are providing single-source message packets of theconfiguration type, compares the height values in data nibbles 70(0) and70(1) (FIG. 5) to NODE HT node height signals whose binary-encoded valuerepresents the level of the control network node 51 in the controlnetwork 14. If the values are the same, the height comparison circuitgenerates a DATA EQ HT (x) data equals height signal ["x" corresponds to"L" (left) or "R" (right)]. On the other hand, if the values differ, aDATA NE HT (x) data not equal to height signal is asserted. The DATA EQHT (x) and DATA NE HT (x) signals are used to control the condition ofthe root flag 1407.

More specifically, if (a) the CL/SS left child single-source signal isasserted contemporaneous with the assertion of the LIP RCV ST 0 signal,indicating that the left child node is providing a single-sourcemessage, and (b) the up output packet assembler 1422 is asserting theOUT PKT SS output message packet single-source and OUT PKT TYPE CONFIGoutput packet type configuration signals, in response to the assertionof the UP RCV ST 3 and LIP RCV ST 4 signals the left height comparisoncircuit 1600 compares the SEL INP DATA (L) selected input data signalsto the NODE HT signals. As shown in FIG. 5, while the LIP RCV ST 3 andUP RCV ST 4 signals are asserted, the SEL INP DATA (L) signals representthe data nibbles 70(0) and 70(1) containing the root height data. If theroot height data in the control network message packet 60 corresponds tothe binary-encoded value of the NODE HT signals, the left heightcomparison circuit 1600 asserts the DATA EQ LT (L) signal.Alternatively, the left height comparison circuit 1600 asserts the DATALT HT (L) signal or DATA GT HT (L) signal if the value of the rootheight data is less than or greater than the binary-encoded value of theNODE HT signals. If the left height comparison circuit 1600 assertseither the DATA LT HT (L) or the DATA GT HT (L) signals, an OR gate 1601asserts the DATA NE HT (L) left data not equal to height signal.

The DATA EQ HT (x) and DATA NE HT (x) signals ("x" references "L" and"R") from the left and right height comparison circuits are coupled to aheight comparison resolver circuit 1602. The height comparison resolvercircuit 1602 includes circuitry for performing two functions. First, aroot establishment enable circuit 1603 asserts a ROOT EN root enablesignal if at least one of the left or right height comparison circuitsasserts the DATA EQ HT (x) signal, and the other heigh comparisoncircuit is not asserting its DATA NE HT (x') signal. This ensures thatthe ROOT EN signal is not asserted if both child nodes are providingsingle-source messages of the configuration packet type, but withdifferent height values. The height comparison resolver circuit alsoincludes a configuration error detector circuit 1604, which asserts aCONFIG S ERR configuration software error signal if at least one of theleft or right height comparison circuits asserts the DATA EQ HT (x)signal, and the other heigh comparison circuit is asserting its DATA NEHT (x') signal.

More specifically, the root establishment enable circuit 1603 includestwo AND gates 1605 and 1606, and an OR gate 1607. If the left heighcomparison circuit 1600 is asserting the DATA EQ HT (L) signal, and theright height comparison circuit is not asserting its DATA NE HT (R)signal, the AND gate 1605 is energized to assert a (L) ROOT EN left rootenable signal. On the other hand, if the right height comparison circuitis asserting the DATA EQ HT (R) signal, and the left height comparisoncircuit 1600 is not asserting its DATA NE HT (L) signal, the AND gate1606 is energized to assert a (R) ROOT EN right root enable signal. Ifeither the (L) ROOT EN signal or the (R) ROOT EN signal is asserted, theOR gate 1607 is energzied to assert the ROOT EN root enable signal.

It will be appreciated that the (x) ROOT EN signal will be asserted if(a) the "x" (left or right) height comparison circuit determines that acontrol network message packet 60 has been received from the "x" childindicating that this control network node 51 should be a root node, and(b) no control network message packet 60 has been received from theother child node indicating that another node should be root node. Ifboth child nodes provide control network message packets 60 indicatingthat the control network node 51 should be a root node, both the ANDgates 1605 and 1606 are energized to assert the respective (x) ROOT ENroot enable signals.

Similarly, the configuration error detector circuit 1604 includes twoAND gates 1610 and 1611 and an OR gate 1612. If the left heighcomparison circuit 1600 is asserting the DATA EQ HT (L) signal, and theright height comparison circuit is also asserting its DATA NE HT (R)signal, the AND gate 1610 is energized to assert a LFT EQ/RT NE S ERRleft equal/right not equal software error signal. On the other hand, ifthe right height comparison circuit is asserting the DATA EQ HT (R)signal, and the left height comparison circuit 1600 is also assertingits DATA NE HT (L) signal, the AND gate 1606 is energized to assert a RTEQ/LFT NE S ERR right equal/left not equal software error signal. Itwill be appreciated that either signal will be asserted if (a) the aheight comparison circuit determines that a control network messagepacket 60 has been received from the its child indicating that thiscontrol network node 51 should be a root node, and (b) another controlnetwork message packet 60 has been received from the other child nodeindicating that another node should be root node. If either the LFTEQ/RT NE S ERR signal or the RT EQ/LFT NE S ERR signal is asserted, theOR gate 1612 is energized to assert the CONFIG S ERR configurationsoftware error signal, which may be used by the up output messageassembler 1422 in establishing the condition of the soft ware error bit76 in a control network message packet 60.

The ROOT EN and CONFIG S ERR signals are used by a root flagconditioning-circuit 1620 to establish the condition of the root flag1407. If the CONFIG S ERR configuration software signal is negated, andIf the up output packet assembler 1422 is asserting the OUT PKT SS andOUT PKT TYPE CONFIG signals, an AND gate 1621 is energized insynchronism with the assertion the LIP RCV ST 5 signal to assert a CONDROOT signal to enable the root flag to be conditioned. As noted above,up output packet assembler 1422 asserts the OUT PKT SS and OUT PKT TYPECONFIG signals, if if it is assembling a message of the single-sourcemessage type and configuration packet type, which, in turn, occurs if atleast one child node is providing such a control network message packet60.

The assertion of the COND ROOT condition root signal enables inputterminals of AND gates 1622 and 1623. The AND gates 1622 and 1623 arealso controlled by the true and complement, respectively, of the ROOT ENsignal. If the COND ROOT signal is asserted, and if the ROOT EN signalis also asserted, AND gate 1622 asserts a SEL SET select set signal,which controls a multiplexer 1624. When the SEL SET signal is asserted,multiplexer 1624 couples an asserted signal as a SET ROOT signal to thedata input terminal of a flip-flop 1625. The asserted SET ROOT signalenables the flip-flop 1625 to be set at the next tick of the NODE CLKsignal to assert a ROOT SET EN root set enable signal.

Thereafter the AND gate 1621 is de-energized, to negate the COND ROOTsignal. The negated COND ROOT signal, in turn, de-energizes AND gate1622 which negates the SEL SET select set signal. Since the AND gate1623 is also de-energized, a SEL CLR select clear signal is alsonegated, enabling the multiplexer 1622 to couple the ROOT SET EN signalas the SET ROOT signal to the data input terminal of the flip-flop 1625.Thus, while the SEL SET and SEL CLR signals are negated, the flip-flop1625 maintains its set condition at successive ticks of the NODE CLKsignal.

The now-asserted ROOT SET EN signal is also coupled to an input terminalof a multiplexer 1626. When an UP RCV ST 12 signal is asserted,multiplexer 1625 couples the ROOT SET EN signal to its output terminalas a ROOT FLAG EN root flag enable signal, which, in turn, is coupled tothe data input terminal of root flag 1407. Since the ROOT SET EN signalis asserted, the ROOT FLAG EN signal will be asserted as well, enablingthe root flag to be set at the next tick of the NODE CLK signal. Thesetting of the root flag 1407 enables it to assert the ROOT UP LAT rootup latch signal, which controlls the packet buffer 1406 (FIG. 12B) andis coupled to the parent node as the ROOT UP signal, After the UP RCV ST12 signal is negated, the multiplexer 1625 couples the ROOT UP LATsignal as the ROOT FLAG EN signal, enabling the root flag 1407 tomaintain its condition at successive ticks of the NODE CLK signal.

On the other hand, if, while the AND gate 1621 is asserting the CONDROOT signal, the root establishment enable circuit 1603 is negating theROOT EN signal, AND gate 1623 is energized to assert the SEL CLR selectclear signal. In that case, the AND gate 1622 will be de-energized,negating the SEL SET select set signal. In that condition, themultiplexer 1624 couples a negated signal as the SET ROOT signal, whichclears flip-flop 1625 at the next tick of the NODE CLK signal, andnegating the ROOT SET EN root set enable signal. As above, when the ANDgate 1621 negates the COND ROOT signal, both AND gates 1622 and 1623will be de-energized, negating the respective SEL SET and SEL CLRsignals, so that multiplexer 1624 will maintain the flip-flop 1625 inthe same condition at successive ticks of the NODE CLK signal.

The now-negated ROOT SET EN signal is also coupled to an input terminalof multiplexer 1626. When an LIP RCV ST 12 signal is asserted,multiplexer 1625 couples the negated ROOT SET EN signal to its outputterminal as a ROOT FLAG EN root flag enable signal, which, in turn, iscoupled to the data input terminal of root flag 1407. Since the ROOT SETEN signal is negated, the ROOT FLAG EN signal will be negated as well,enabling the root flag to be cleared at the next tick of the NODE CLKsignal. The clearing of the root flag 1407 enables it to negate the ROOTUP LAT root up latch signal. After the UP RCV ST 12 signal is negated,the multiplexer 1625 couples the ROOT UP LAT signal as the ROOT FLAG ENsignal, enabling the root flag 1407 to maintain its condition atsuccessive ticks of the NODE CLK signal.

As noted above, the multiplexer 1625 couples the ROOT SET EN root setenable signal to the data input terminal of root flag 1407 as the ROOTFLAG EN signal in synchronism with assertion of the UP RCV ST 12 signal.Thus, the root flag 1407 is conditioned contemporaneous withtransmission by the up output packet assembler 1422 of the end of thecontrol network message packet 60 it is currently transmitting. Sincethe ROOT UP LAT signal provided by the root flag 1407 controlstransmission of control network message packets 60 to the parent nodeand also controls storage of packets 60 in the packet buffer 1406, itwill be appreciated that conditioning of the root flag in synchronismwith assertion of the UP RCV ST 12 signal ensures that partial packetsnot be transmitted to the parent node or stored in the packet buffer.

4. flick down control portion 1402

FIG. 12D depicts a detailed block diagram of the flick down controlportion 1402. With reference to FIG. 12D, the flick down control poriton1402 includes a down source select circuit 1650 that receives P FLICK DN(4:0) signals from a parent node in the control network 14 and BUF PFLICK UP (4:0) signals from the packet buffer 1406 (FIG. 12B), alongwith the ROOT UP LAT signal. In response to the ROOT UP LAT signal, atsuccessive ticks of the NODE CLK signal the down source select circuit1650 selectively couples either the P FLICK DN (4:0) signals or the BUFP FLICK UP (4:0) signals as SEL DN INP DATA (3:0) selected down inputdata signals and a SEL DN INP TAG signal. In addition, the down sourceselect circuit 1650 provides DN INP STA/CTRL down input status/controlsignals and the SEL DN INP TAG signal to a down control circuit 1651.

In turn, the down control circuit 1651, like up control circuit 1424,generates a number of control and timing signals for controlling theoperations of the flick down control portion. In particular, the downcontrol circuit generates DN RCV ST (12:0) down receive state timingsignals, which comprise thirteen signals DN RCV ST 0 through DN RCV ST12, which are successively asserted, in response to successive ticks ofthe NODE CLK signal, in synchronism with the receipt, by the down sourceselect circuit 1650, of the thirteen flicks of a control network messagepacket 60.

In addition, if the root flag 1407 is asserting the ROOT UP LAT root uplatch signal, the down control circuit provides the PACKET BUF RE packetbuffer read enable signal to enable the packet buffer 1406 (FIGS. 12A,12B) to generate BUF P FLICK UP buffered flick up signals representing acontrol network message packet 60. Similarly, If the control networkmessage packet 60 received by the down source select circuit, eitherfrom the parent node in the control network 14 or from the packet buffer1406, is a multiple-source message, the down control circuit 1651provides a SCAN BUF RE scan buffer read enable signal, which enables thescan buffer (FIGS. 12A, 12B) to couple message packet data storedtherein to the flick down control portion 1402. The data from the scanbuffer 1410 is combined with message packet data from the the messagepacket 60 being received by the down source select circuit 1650 by aflick down data processor 1652.

The down control circuit 1651 also receives the SEL DN INP TAG selecteddown input tag signal from the down source select circuit 1650 and, inresponse to the signals representing the condition of the scan flow bits72(i) (Fib 5), generates FLOW CTRL (UP) flow control up signals that arecoupled to the up control circuit 1424. As described above, the FLOWCTRL (UP) signals are used by the flick up control portion to controlthe transfer of control network message packets 60 of themultiple-source message type to the parent node. Similarly, the downcontrol circuit 1681 receives a FLOW CONTROL (DN) flow control downsignal from the up control circuit, and uses it to control the transferof control network message packets 60 of the multiple-source messagetype to the child nodes.

Furthermore, the donw control circuit 1651 receives an ERR error signalfrom the up control circuit 1424. The down control circuit may use theERR signal to enable the conditioning of the software error bit 76 incontrol network message packets 60 transmitted by the flick down controlportion 1402.

The flick down control portion 1402 includes left and right down outputpacket assembler circuits 1653 and 1654 which receive signals from thedown source select circuit 1650, from the flick down data processor1652, from a down tag processor 1655 and, under control of C(x) DN OUTSEL child down output select signals ["x" references "L" (left) or "R"(right)] from the down control circuit 1651, generates control networkmessage packets 60 for transfer to the respective left and right childnodes in the control network 14. In particular, at successive ticks ofthe NODE CLK signal the down output packet assembler circuits 1653 and1654 generate C(x) FLICK DN signals representing successive flicks ofcontrol network message packets 60 generated thereby.

The types of control network message packets 60 generated by the downoutput packet assemblers 1653 and 1654 depends in part on the conditionof a (x) ROOT UP signal as controlled by the child node connectedthereto. In particular, in response to the (x) ROOT UP signal, from achild node, shifting from the negated condition to an assertedcondition, the corresponding down output packet assembler circuit, atthe end of the control network message packet 60 it is then transmittingbegins transmitting C(x) FLICK DN signals representing message packetsof the NPAC nil packet message type. The down packet assembler circuitwaits until the end of the packet it is transmitting when the conditionof the (x) ROOT UP signal changes, so that the flick down controlportion 1402 of the child node receives the entire control networkmessage packet 60. Similarly, if the condition of the (x) ROOT UP signalchanges from an asserted condition to a negated condition, the downpacket assembler circuit continues transmitting NPAC messages to thechild node until the beginning of the next packet enabled by the downcontrol circuit 1651. Accordingly, the down output packet assemblersgenerate and transfer complete message packets to their respective childnodes.

Otherwise the types of control network message packets 60 generated bythe respective down output packet assembler circuits 1653 and 1654 atany particular time depends upon the type of control network messagepacket 60 being received by the down source select circuit 1650. inparticular, if the down source select circuit 1650 receives a packet 60of the single-source message type or idle message type, the down controlcircuit 1651 enables the down output packet assembers to use thelow-order four bits of each flick, comprising packet information portion(FIG. 5) thereof in generating single-source message packets fortransfer to the child nodes. In generating the high-order tag bits forthe successive flicks of the message packet being generated, the downoutput packet assembler circuits use signals from various sources,including the down tag processor 1655 and the down control circuit 1651,to determine the condition of the respective bits.

Similarly, if the down source select circuit 1650 receives a messagepacket of the multiple-source type, the down control circuit 1651 willenable down output packet assembler circuits to use either the SEL DNINP DATA selected down input data, the SCAN BUF DATA scan buffer datasignals or DN PROC DATA down processed data signals from the flick downdata processor 1652 in generating the data portion 62 of a controlnetwork message packet 60. The PROC FLICK (DN) DATA processed flick downdata signals from down data processor 1652 represent the sum, logicalOR, logical XOR and maximum of the SEL DN INP DATA signals, as generatedby an adder 1660, an OR circuit 1661, an XOR circuit 1662 and acomparator 1663, respectively. If, for example, the multiple-sourcemessage enables a scan forward operation, the down control circuit 1651enables the down output packet assembler circuit 1653 to use the SCANBUF DATA in the data portion 62 in the packet 60 being transmitted tothe left child. In addition, the down control circuit 1651 enables thedown output packet assembler circuit 1654 to use the DN PROC DATA downprocessed data signals from the flick down data processor 1652 in thepacket 60 being transmitted to the right child.

On the other hand, if the multiple source message enables a scanbackward operation, the down control circuit 1651 enables the downoutput packet assembler circuit 1653 to use the DN PROC DATA downprocessed data signals from the flick down data processor 1652 in thepacket 60 being transmitted to the left child. In addition, the downcontrol circuit 1651 enables the down output packet assembler circuit1654 to use the SCAN BUF DATA in the data portion 62 in the packet 60being transmitted to the right child. Effectively, if a multiple-sourcemessage enables a scan backward operation, the down control circuit 1651enables the down output packet assembler circuits 1653 and 1654 toreverse the message packets transmitted thereby to their respectivechild nodes, to achieve the reverse scan operation as described above.

Furthermore, if the multiple source message enables a reduce operation,the down control circuit enables the down output packet assemblercircuits 1653 and 1654 to use the SEL DN INP DATA signals representingthe flicks of the packet data portion 62 in the message packets 60assembled thereby. As described above, in a reduce operation, thecombination of the data from the respective leaves 21 (FIGS. 1, 4A and4B) is performed by the flick up control portions 1401 as the controlnetwork message packets 60 are being transmitted up the control network14 to the root node, and the packets transmitted down carry the resultsof the reduce operation as determined by the root node.

To enable generation of portions of a multiple source message other thanthe packet data portion 62, the down control circuit 1651 may enable thedown output packet assembler circuits to use SEL DN INP DATA signalsreceived by the down source select circuit 1650 representing thoseportions of the message. For example, the down control circuit 1651 mayenable the down output packet assembler cirucits to use SEL DN INP DATAsignals representing the first three flicks, comprising the packetheader 61, in the message packets generated thereby for transmission tothe child nodes. In addition, each down output packet assembler circuit1653 and 1654 has a checksum generator that generates a checksum valuefor use in the flick representing the checksum field 63 of the controlnetwork message packet 60 being generated.

Finally, if the FLOW CTRL (UP) signal from the up control circuit 1681indicates that a child node is unable to receive message packets 60, ifa multiple source message is thereafter received from the parent node,the down control circuit 1681 enables the left and right down outputpacket assembler circuits 1653 and 1654 to generate idle messages fortransfer to their respective child nodes. As will be described below inconnection with FIG. 12D-1, the down source select circuit 1650 willbuffer the multiple-source message received from the parent node, andthe down control circuit 1651 will enable the up control circuit 1424to, in turn, enable the up output packet assembler to set the scan flowbits 72 of control network message packets 60 it is transmitting to theparent node. The parent node is thereafter inhibited from transmittingmultiple-source messages to the flick down control portion 1402.

Many of the circuits of the flick down control portion 1402 are similarto corresponding circuits of the flick down control portion 1401 andwill not be described. FIG. 12D-1 depicts details of the down sourceselect circuit 1650, showing the selection of either the P FLICK DN(4:0) signals from the parent node or the BUF P FLICK UP (4:0) signalsfrom the flick up control portion 1401 as the SEL DN INP DATA (3:0) andSEL DN INP TAG signals. As noted above, the down source select circuit1650 makes the selection in resposne to the ROOT UP LAT root up latchsignal from the root flag 1407 (FIGS. 12A and 12C).

With reference to FIG. 12D-1, the down source select circuit 1650includes a down source selector circuit 1670 that selectively coupleseither the P FLICK DN parent flick down signals or the BUF P FLICK UPbuffered parent flick up signals, or alternatively PARKED P FLICK parkedparent flick signals from a park buffer 1671 as the SEL DN INP DATA(3:0) signals and the SEL DN INP TAG signals. The park buffer 1671 canbuffer a control network message packet 60 of the multiple-sourcemessage type if a child node is disabling transfer of such messagesthereto.

The down source selector circuit 1670 is controlled by two circuits,namely, a parent/packet buffer select enable circuit 1672 and a parkbuffer select enable circuit 1673. Both circuits 1672 and 1673 operatein connection with message type identification signals from a downpacket type decoder 1674, which, in synchronism with the DN RCV ST RSTand DN RCV ST 0 down receive state reset and zero timing signals fromthe down control circuit 1651, generates signals identifying the messagetype of the message packet 60 being coupled by the down source selectorcircuit 1670. In particular, the down packet type decoder generatesP/NPAC, P/MS, P/SS, P/IDLE and P/ABS signals which, when asserted,indicate that the message packet 60 is of the NPAC nil packet,mulitple-source, single-source, idle or abstain types, respectively.

The parent/packet buffer select enable circuit 1672 generates ahigh-order SEL PAR/ROOT select parent or root signal for controlling thedown source selector 1670. The circuit 1672 operates in resposne to ROOTUP LAT root up latch and ROOT SET EN root set enable signals tocondition a flip-flop 1675 to, in turn, control the SEL PAR/ROOT signal.If the parent node is transmitting P FLICK DN signals representing NPACnil packet messages, the down packet type decode circuit 1674 willassert the P/NPAC signal in synchronism with the DN RCV ST 0 signal. Inthat condition, an AND gate 1676 is de-energized, which disables oneinput terminal of an OR gate 1677. Since at this point the DN RCV ST10-12 signals, comprising the DN RCV ST 10 through DN RCV ST 12 rceivestate timing signals, are also negated, the OR gate is de-energized,negating a SEL ROOT SRCE select root source signal. Since the DN RCV ST1-9 signals, comprising the DN RCV ST 1 through DN RCV ST 9 signals, arealso negated, a SEL ROOT SRCE HOLD select root hold signal is alsonegatd, enabling a multiplexer 1680 to couple the ROOT SET EN root setenable signal from flip-flop 1625 of the root flag conditioning circuit1620 (FIG. 12C) to the data input terminal of flip-flop 1675.

If the ROOT SET EN signal is negated, flip-flop 1675 is clear at thenext tick of the NODE CLK signal, thereby negateing the SEL PAR/ROOTsignal. In that case, the down source selector 1670 couples the P FLICKDOWN signals from the parent node, or PARKED P FLICK signals from thepark buffer 1671, as the SEL DN INP DATA (3:0) selected down input datasignals and the SEL DN INP TAG signal. When the DN RCV ST 1-9 signalsare asserted, the multiplexer 1680 couples the SEL PAR/ROOT signal fromthe output terminal of flip-flop 1675 back to its data input terminal,thereby maintaining the flip-flop 1675 in the clear condition, and theSEL PAR/ROOT signal negated, at successive ticks of the NODE CLK signal.

On the other hand, if the ROOT SET EN signal is asserted, flip-flop 1675is set at the next tick of the NODE CLK signal, thereby asserting theSEL PAR/ROOT select parent/root signal. It will be appreciated that thiswill be contemporaneous with the setting of root flag 1407 in responseto the asserted ROOT SET EN signal. The assertion of the SEL PAR/ROOTsignal enables the down source selector 1670 to couple the BUF P FLICKUP signals from the packet buffer 1410, or the PARKED P FLICK signals asthe SEL DN INP DATA (3:0) and SEL DN INP TAG signals. Thus, if theparent node is transmitting NPAC nil packet messages to the controlnetwork node 51, the parent/packet buffer select enable circuit 1672enables the down source selector 1670 to couple BUF P FLICK UP signalsfrom the packet buffer 1406, or the PARKED P FLICK signals from the parkbuffer, immediately upon the node becoming root node.

The assertion of the DN RCV ST 1-9 signals enables the multiplexer 1680to couple the asserted SEL PAR/ROOT signal to the input terminal offlip-flop 1675, enabling it to maintain its set condition.

Similarly, if the parent node is transmitting NPAC nil packet messagesto the control network node 51, when the DN RCV ST 0 signal is assertedan AND gate 1681 is energized, enabling an OR gate 1682 to assert a TSTROOT UP test root up signal. The asserted TST ROOT UP signal enables amultiplexer 1683 to couple the ROOT UP LAT root up latch signal fromroot flag 1407 to the data inpu terminal of a flip-flop 1684. If theroot flag 1407 is clear and the ROOT UP LAT signal is negated,indicating that the control network node 51 is not a root node, theflip-flop 1684 is reset at the next tick of the NODE CLK signal. On theother hand, if the root flag is set and the ROOT UP LAT signal isasserted, the flip-flop 1684 will be set, asserting a ROOT DN LAT rootdown latched signal.

On the other hand, if the parent node is not transmitting NPAC nilpacket messages to the control network node 51 when the DN RCV ST 0signal is asserted, the OR gate 1682 will assert the TST ROOT UP signalin rseponse to the assertion of the DN RCV ST 12 signal, which iscontemporaneous with P FLICK DN signals representing the last flick ofthe control network message packet 60 currently being received. At thatpoint, the multiplexer 1683 will couple the ROOT LAT UP signal to thedata input terminal of flip-flop 1684. If the ROOT LAT UP signal isasserted, the flip-flop 1684 will be set at the next tick of the NODECLK signal to assert the ROOT DN LAT root down latched signal.Alternatively, if the ROOT UP LAT signal is negated, the flip-flop 1684will be cleared to negate the ROOT DN LAT signal.

It will be appreciated that, after the DN RCV ST 12 signal is laternegated, enabling the OR gate 1683 to negate the TST ROOT UP signal, themultiplexer 1683 couples the ROOT DN LAT to the data input terminal offlip-flop 1684 to maintain the flip-flop 1675 in its condition atsuccessive ticks of the NODE CLK signal. Effectively, if the parent isnot transmitting NPAC nil packet messages, the multiplexer 1683 enablesthe flip-flop 1684 to be conditioned in response to the ROOT UP LATsignal from root flag 1407 when the DN RCV ST 12 signal, which iscontemporaneous SEL DN INP DATA and SEL DN INP TAG signals representingthe last flick of the control network message packet 60 being receivedby the down source selector 1670.

Regardless of the condition of the ROOT DN LAT signal, the OR gate 1677is contemporaneously energized to assert the SEL ROOT SRC signal. Thus,the multiplexer 1683 couples the ROOT DN LAT signal to the data inputterminal of flip-flop 1675. If the ROOT UP LAT signal is negated,indicating that the control network node 51 is not a root node, theflip-flop 1675 will be cleared to negate the SEL PAR/ROOT signal. Inthat condition, the down source selector 1670 couples the P FLICK DOWN(4:0) signals, or the PARKED P FLICK signals from the park buffer 1671,as the SEL DN INP DATA (3:0) and SEL DN INP TAG signals. Alternatively,if the ROOT UP LAT signal is asserted, indicating that the controlnetwork node 51 is a root node, the flip-flop 1675 will be set to assertthe SEL PAR/ROOT signal. In that condition, the down source selector1670 couples the BUF P FLICK DOWN (4:0) signals, or the PARKED P FLICKsignals from the park buffer 1671, as the SEL DN INP DATA (3:0) and SELDN INP TAG signals.

The park buffer select enable circuit 1673 generates a SEL PKD PKTselect parked packet signal that, when asserted enables the down sourceselector circuit 1670 to couple the PARKED P FLICK signals from the parkbuffer 1671 as the SEL DN INP DATA (3:0) and SEL DN INP TAG signals. If,when the DN RCV ST 0 signal is asserted, the down packet type decoder isasserting either the P/IDLE or the P/ABS signals, an OR gate 1690 isenergized to enable an AND gate 1691. If the DN RCV ST 0 is concurrentlyasserted, the message packet message packet 60 being coupled by the downsource selector 1670 is of the idle or abstain message type. If, inaddition, a PKD DN PKT parked down packet signal is being asserted bythe park buffer 1671, indicating that the park buffer 1671 contains aparked message packet, the AND gate 1691 is energized to assert a SEL PKBUF signal.

The asserted SEL PK BUF signal enables a multiplexer 1692 to couple anasserted signal to the data input terminal of a flip-flop 1693. Theflip-flop 1693 is set at the next tick of the NODE CLK signal, therebyasserting the SEL PKD PKT select parked packet signal.

When the DN RCV ST 0 signal is negated, the AND gate 1691 isde-energized, to negate the SEL PK BUF signal. At that point, however,the DN RCV ST 1-9 signals, comprising the DN RCV ST 1 through DN RCV ST9 signals are asserted, which comprise a HOLD PK BUF hold park buffersignal. The asserted HOLD PK BUF signal enables the multiplexer 1692 tocouple the asserted SEL PKD PKT select parked packet signal to the datainput terminal of flip-flop 1693, to enable the flip-flop to maintainits state during the successive ticks of the NODE CLK signal.

When the DN RCV ST 10 signal is later asserted, both the SEL PK BUF andthe HOLD PK BUF signals will be negateed, enabling the multiplexer 1692to couple a negated signal to the data input terminal of the flip-flop1693. The flip-flop 1693 is reset at the next tick of the NODE CLKsignal, thereby negating the SEL PKD PKT select parked packet signal. Atthat point, the down source selector 1670 couples the P FLICK DN or BUFFLICK DN signals, as determined by the condition of the SEL PAR/ROOTselect parent or root signal, as the SEL DN INP DATA (3:0) and SEL DNINP TAG signals.

The down control circuit 1651 can also enable a control network messagepacket 60 to be parked in the park buffer 1671, in a manner similar tothe parking of a packet 60 in the park buffer 1442 in the left and rightchild receiver/buffers 1420(x). If, while the FLOW CTRL (DN) signalsfrom the flick up control portion 1401 indicates that a child node isunable to receive additional multiple-source messages, the down sourceselect circuit 1650 receives a message packet. 60 of the multiple-sourcemessage type, the down control circuit 1651 asserts a PARK EN parkenable signal that enables a multiplexer 1694 to couple SEL DN INP DATA(3:0) signals to data input terminals of the park buffer 1671. The parkbuffer 1671 latches the signals at the successive ticks of the NODE CLKsignal to buffer the packet. After the packet 60 is buffered, the downcontrol circuit 1651 negates the PARK EN signal, which couples theoutput of the park buffer 1671 to its input terminals. After a messagepacket 60 is buffered in the park buffer 1671, the down control circuit1651 conditions the FLOW CTRL (UP) signals to, in turn, enable the flickup control portion 1401 to provide scan flow bits 72(i) (FIG. 5) todisable the parent node from transmitting packets 60 of the multiplesource type thereto.

E. Diagnostic Network 1. General

FIG. 13A is a general block diagram of a diagnostic network node100(h,p,r-1) used in the data router described above, and FIGS. 13B-1through 11C comprise detailed block and logic diagrams of the diagnosticnetwork node 100(h,p,r-I). With reference to FIG. 13A, the diagnosticnetwork node, which will be generally identified by reference numeral100, includes an address token/data control portion 2000 and a test datacontrol portion 2001. The address token/data control portion 2000generally corresponds to the address control circuit 102 (FIG. 6A) andthe test data control portion 2001 generally corresponds to the datacontrol portion 103 (FIG. 6A).

The diagnostic network node 100 receives PAR ADRS CTRL parent addresscontrol signals from a parent node, or from the diagnostic processor 101(FIG. 6A) at one set of data input terminals of amultiplexer/demultiplexer 2002. The multiplexer/demultiplexer 2002includes another set of data input input terminals, which receive acorresponding set of DP ADRS CTRL diagnostic processor signals over abus 2003 from a local diagnostic processor (not shown). The localdiagnostic processor also generates a P SEL parent select signal, whichcontrols the transfer of signals between buses 2003 or 104(P) and a bus2004 connected to the address token/data control portion 2000. The localdiagnostic processor may negate the P SEL signal to enable themultiplexer/demultiplexer 2002 to couple the address control signalsbetween bus 104(P) and the bus 2004, to thereby enable the signals to betransferred between the parent node 100 or the diagnostic processor 101and the address token/data control circuit 2000. Alternatively, thelocal diagnostic processor may assert the P SEL signal to enable themultiplexer/demultiplexer 2002 to couple the address control signalsbetween the bus 2003 and bus 2004 to thereby enable the signals to betransferred between the local diagnostic processor and the addresstoken/data control circuit 2000.

The diagnostic network node 100 also receives PAR DATA parent test datasignals over bus 110(P) from a parent node or from the diagnosticprocessor 101 (FIG. 6A) at one set of data input terminals of amultiplexer/demultiplexer 2005. The multiplexer/demultiplexer 2005includes another set of data input input terminals, which receive acorresponding set of DP DATA diagnostic processor data signals over abus 2006 from a local diagnostic processor (not shown). The P SEL parentselect signal also controls the transfer of signals between buses 110(P)or 2006 and a bus 2007 connected to the test data control portion 2001.The local diagnostic processor may negate the P SEL signal to enable themultiplexer/demultiplexer 2005 to couple the test data signals betweenbus 110(P) and the bus 2007, to thereby enable the signals to betransferred between the parent node 100 or the diagnostic processor 101and the test data control circuit 2001. Alternatively, the localdiagnostic processor may assert the P SEL signal to enable themultiplexer/demultiplexer 2005 to couple the test data signals betweenthe bus 2006 and bus 2007 to thereby enable the signals to betransferred between the local diagnostic processor and the test datacontrol circuit 2001.

It will be appreciated that, if the node 100 comprises the root node100(M,0,0 . . . 0) in the height decoding tree (FIGS. 6A through 6C),the diagnostic processor 101 may be connected to either buses 104(P) and110(P), respectively, or to buses 2003 and 2006. If the diagnosticprocessor 101 is connected to buses 104(P) and 110(P), it will maintainthe P SEL parent select signal negated, and if it is connected to buses2003 and 2006 it will maintain the P SEL signal asserted. Alternatively,diagnostic processors may be connected both to buses 104(P) and 110(P),on the one hand, and to buses 2003 and 2006, on the other hand, and theP SEL signal may be controlled to enable coupling of signals betweenthere and buses 2004 and 2007.

The address token/data control portion 2000 transmits signals to, andreceives signals from, the various child diagnostic network nodes 100connected thereto over buses 104(C_(i)). In the embodiment depicted inFIGS. 13A through 13C, the diagnostic network node 100 may be connectedto "m" child nodes, each over a separate bus 104(C_(i)). Similarly, thetest data control portion 2001 transmits signals to, and receivessignals from, "m" child diagnostic network nodes 100 connected theretoover buses 110(C_(i)), each over a separate bus 110(C_(i)).

As also shown in FIG. 13A, the address token/data control poriton 2000includes the flags 106(C_(i)). Each flag 106(C_(i)) controls an EN(i)enabling signal that controls transfer by the diagnostic network node100 over the buses 104(C_(i)) and 110(C_(i)) in tandem. In particular,the flag 106(C_(i)), when set, enables the address token/data controlcircuit 2000 to transmit and receive signals over the bus corresponding104(C_(i)). In addition, the flag 106(C_(i)), when set, enablesassertion of the corresponding EN(i) enabling signal that, in turn,enables the test data control portion 2001 to transmit and receivesignals over the associated bus 110(C_(i)).

Before proceeding further, it would be helpful to describe the varioussignals transmitted over the buses 2002 and 2004. Bus 2002 compriseslines for carrying six signals, five of which, namely, lines 2010through 2014, are received by the address token/data control portion2000. The bus 2002 includes a sixth line 2015 for carrying a signalgenerated by the address token/data control poriton 2000 up the treedefining diagnostic network 16 to the parent node or to the diagnosticprocessor connected to multiplexer/demultiplexer 2003, depending on thecondition of the P SEL signal. Buses 104(C_(i)) connected between theaddress token/data control portion 2000 and child diagnostic networknodes have lines for carrying similar signals therebetween.

In particular, bus 2002 includes a line 2010 that carries an ACLK (P)address clock from parent signal, which the diagnostic network node 100uses as a clock signal to synchronize operations in the addresstocken/data control portion 2000 in connection with other signalscomprising bus 2004. In addition, a line 2011 carries an AMS (P) addressmode select from parent signal, which controls a control circuit in theaddress token/data control circuit 2000. The node transmits the ACLK (P)signal and AMS (P) to all of its children.

A line 2013 carries an ATI (P) address token in from parent signal, anda line 2014 carries an ADI (P) address data in from parent signal, bothof which cooperate to sequentially condition the flags 106(C_(i)) in thediagnostic network node 100. The conditioning of flags 106(C_(i)) in theaddress token/data control 2000 is controlled by a token, which isshifted through a shift register (described below in connection withFIG. 13B-1) in the address token/data control portion 2000. The shiftregister has a number of stages, each corresponding to one of the flags106(C_(i)) in the diagnostic network node 100. When the token is in astage in the shift register associated with a particular flag106(C_(K)), if the ADI (P) signal is asserted, the flag is set at thenext tick of the ACLK (P) address clock signal. On the other hand, ifthe ADI (P) Signal is negated, the flag 106(CK) is cleared.

The condition AMS (P) address mode select signal, along with the ticksof the ACLK (P) signal, controls shifting of the token through the shiftregister. After the token has shifted through the shift register on thediagnostic network node 100, it shifts out and is transmitted over thebuses 104(C_(i)) associated with those of the flags 106(C_(i)) that areset to the nodes connected thereto. The node also transmits the ADI (P)signal to all of its children. Accordingly, it will be appreciated thatthe conditioning of the flags 106(C_(i)) in each of the child nodeswhich receive the token will be accomplished in parallel, with theconditions of the flags 106(C_(i)) in the respective child nodes beingcontrolled in parallel by the condition of the ADI (P) signal inresponse to the next tick of the ACLK address clock signal.

In addition, after a flag 106(C_(i)) is conditioned, the flag's statemay be retrieved. Retrieval is enabled under control of the AMS (P)signal, and the state is represented by the condition of an ADO (P)address data out to parent signal over a line 2015. If addresstoken/data control circuit 2000 of node 100 receives ADO (C_(i)) addressdata out signals from one or more of its child nodes, associated withset flags 106(C_(i)), the address token/data control circuit 2000 maycombine them under control of an EADO (P) expected address data out fromparent signal on a line 2012 in bus 2004. If the ADO (C_(i)) signals areexpected to be asserted, the EADO (P) signal enables the addresstoken/data control circuit 2000 to logically AND them together. In thatcase, if the ADO (C_(i)) signals from child nodes associated with theset flags 106(C_(i)) are all asserted, the ADO (P) address data out toparent signal will be asserted, but if one of the ADO (C_(i)) signals isnegated the ADO (P) signal will be negated. On the other hand, if theADO (C_(i)) signals from the child nodes are expected to be negated, theEADO (P) signal enables the address token/data control circuit 2000 tologically OR them together. In that case, if the ADO (C_(i)) signalsfrom child nodes associated with the set flags 106(C_(i)) are allnegated, the ADO (P) address data out to parent signal will be negated,but if one of the ADO (C_(i)) signals is asserted the ADO (P) signalwill also be asserted.

Bus 2007 comprises lines for carrying five signals, four of which,namely, lines 2020 through 2023, are received by the test data controlportion 2001. The bus 2007 includes a fifth line 2024 for carrying asignal generated by the test data control portion 2007 up the treedefining diagnostic network 16 to the parent node or to the diagnosticprocessor connected to multiplexer/demultiplexer 2005, depending on thecondition of the P SEL signal. Buses 104(C_(i)) connected between theaddress token/data control portion 2000 and child-diagnostic networknodes have lines for carrying similar signals therebetween.

As noted above, the interface between leaf nodes in the diagnosticnetwork 16 and each pod in one embodiment corresponds to the JTAG("Joint Test Action Group") interface, as described in IEEE Std. 1149.1(hereinafter "JTAG specification"). The JTAG interface comprises foursignals including a TCK test clock signal, a TMS test mode signal and aTDI test data in signsl, all of which are provided to the pod by theleaf node 100, and a TDO test data out signal provided by the pod to theleaf node 100. The use of the signals is defined in the aforementionedJTAG documentation. Generally, the TCK signal operates as a clocksignal, the TMS test mode select signal operates as a test controlsignal, and the TDI signal defines test data. The TDO signal definestest results.

With this background, the test data; control portion 2001 in each node100 receives a TCLK (P) test clock from parent signal, a TMS (P) testmode select from parent signal, and a TDI (P) test data in from parentsignal on lines 2020, 2021 and 2022, respectively of bus 2007. The testdata control portion 2001 couples these signals onto respective lines inthose of buses 110(C_(i)) associated with the asserted EN (i) enablesignals. As noted above, those of EN(i) enable signals that are assertedcorresponds to those of flags 106(C_(i)) that are set. The signals arethus passed from respective parent node to respective child nodes downthe paths defined by the set flags 106(C_(i)) to the selected ones ofthe pods.

In addition, bus 2007 includes a line 2023 for carrying an ETDO (P)expected test data out from parent signal. The test data control portion2001 couples this signal to child nodes along with the TDI (P) and othersignals on lines 2020 through 2022. The bus 2007 also includes a line2024 for carrying a TDO (P) test data out to parent signal, whoseutility will be made clear in the following.

The leaf nodes 100 in the diagnostic network 16 provide the TCLK(C_(i)), TMS (C_(i)) and TDI (C_(i)) signals to the respective pods asthe TCK, TMS and TDI signals, as called for by the aforementioned JTAGspecification. In response, the pods provide a TDO test data out signal,which is also called for by the aforementioned JTAG specification. TheTDO signal is received by the test data control portion 2001 as a TDO(C_(i)) signal in the pod's bus 110(C_(i)). The test data controlportion 2001 of a leaf node 100 receives the TDO (C_(i)) signals fromall of the pods associated with asserted EN(i) enable signals, andcombines them as called for by the ETDO (P) expected test data out fromparent signal.

The test data control portion 2001 uses the ETDO (P) signal inconnection with the TDO (C_(i)) signals that are associated withasserted EN(i) signals in the same way the address token/data controlportion 2000 uses the EADO (P) expected address data out signal inconnection with the ADO (C_(i)) signals associated with set flags106(C_(i)). If the test data control circuit 2001 receives ADO (C_(i))address data out signals from one or more of the pods or child nodesconnected thereto that are associated with asserted EN(i) signals, thetest data control circuit 2001 may combine them under control of theETDO (P) signal. If the TDO (C_(i)) signals are expected to be asserted,the ETDO (P) signal enables the test data control circuit 2001 tologically AND them together. In that case, if the TDO (C_(i)) signalsfrom child nodes associated with the asserted EN(i) signals are allasserted, the TDO (P) test data out to parent signal will be asserted,but if one of the TD0 (C_(i)) signals is negated the TDO (P) signal willbe negated. On the other hand, if the TDO (C_(i)) signals from the childnodes are expected to be negated, the ETDO (P) signal enables the testdata control circuit 2001 to logically OR them together. In that case,if the TDO (C_(i)) signals from child nodes associated with the assertedEN(i) signals are all negated, the TDO (P) test data out to parentsignal will be negated, but if one of the TDO (C_(i)) signals isasserted the TDO (P) signal will also be asserted.

2. Address Token/Data Control Portion 2000

The address token/data control portion 2000 will be described inconnection with FIGS. 13B-1 and 13B-2. With reference to FIG. 13B-1, theaddress token/data control portion 2000 includes a flag register 106 anda token register 2030. The flag register 106 comprises a set offlip-flops each corresponding to one of flags 106(C_(i)). The addresstoken/data control portion 2000 depicted in FIG. 13B-1 includes "m"flags 106(C_(i)), identified by reference numerals 106(C₀) through106(C_(m-1)). Each flag 106(C_(i)) generates a corresponding EN(i)enable signal.

The token register 2030 includes a like number of stages 2030(1) through2030(m-1) [generally identified by reference numeral 2030(i)] connectedto a like number of multiplexers 2032(i) that together form a shiftregister. Each token register stage 2030(i), in turn, controls amultiplexer 2031(i) that controls the source of signals provided to theinput terminal of the corresponding flag 106(C_(i)) in flag register106. In particular, the multiplexers 2031(i) have one data inputterminal that receives the ADI (P) address data in signal from line2014, and a second data input terminal that receives the EN(i) enablesignal output by the respective flag 106(C_(i)). If the correspondingtoken register stage 2030(i) is asserting a TR (i) token registersignal, the multiplexer 2031(i) couples the ADI (P) signal to the datainput terminal of the flag 106(C_(i)). The flag 106(C_(i)) latches theADI (P) signal at the next tick of the ACLK (P) address clock fromparent signal. On the other hand, if the corresponding stage of thetoken register stage 2030(i) is not asserting the TR (i) token registersignal, the multiplexer 2031(i) couples the EN (i) enable signal outputby the flag 106(C_(i)) to the data input terminal of the flag, whichlatches it at the next tick of the ACLK (P) signal. Accordingly, thecondition of the TR (i) signal from the respective token register stage2030(i) determines whether state of the corresponding flag 106(C_(i))remains the same or whether it is controlled by the ADI (P) signal atthe next tick of the ACLK (P) signal.

The multiplexers 2032(i) are controlled by a SHIFT TOKEN signal from acontrol circuit 2033. The SHIFT TOKEN signal enables a token,represented by a set token register stage 2030(i), resulting in anasserted TR (i) signal, to be shifted from the first token registerstage 2030(0) to the last stage 2030(m-1), in response to the successiveticks of the ACLK signal. The receipt of the token by the first tokenregister stage 2030(0) is reprsented by the assertion of the the ATI (P)address token in from parent signal when the control circuit 2033asserts the SHIFT TOKEN signal. The ATI (P) signal is coupled to onedata input terminal of the multiplexer 2032(0) connected to the firststage 2030(0) of token register 2030. The second data input terminal ofthe multiplexer 2032(0) is connected to receive the TR(0) token registersignal output by the stage, 2030(0).

If the control circuit 2033 is asserting the SHIFT TOKEN signal, themultiplexer 2032(0) couples the ATI (P) signal to the intput terminal ofthe token register stage 2030(0), which latches the signal at the nexttick of the ACLK (0) signal. If the ATI (P) signal is negated, the stage2030(0) is cleared, which, in turn, enables the stage to negate the TR(0) signal. On the other hand, if the ATI (P) signal is asserted, whichindicates that the parent diagnostic network node or the diagnosticprocessor is transmitting the token to this node 100, the stage 2030(0)is set, which, in turn, enables the stage to assert the TR (0) signal.

The series of multiplexers 2032(i) are controlled in unison by the SHIFTTOKEN signal. Thus, if a token register stage 2030(i) is set, indicatingthat that stage 2030(i) has the token, if the control circuit 2033 isasserting the SHIFT TOKEN signal the multiplexer 2030(i+1) is enabled tocouple the TR (i) signal to the input of its respective token registerstage 2030(i+1 ), where it is latched at the next tick of the ACLK (P)signal. If the token register stage 2030(i) is set, asserting its TR (i)signal, the stage 2030(i+1) will be set to assert its TR (i+1) signal.Similarly, if the token register stage 2030(i) is clear, negating its TR(i) signal, the stage 2030(i+1) will be clear. Thus, while the controlcircuit 2033 asserts the SHIFT TOKEN signal, the token register stages2030(i) and multiplexers 2032(i) effectively shift the token atsuccessive ticks of the ACLK (P) signal.

On the other hand, if the control circuit 2033 is negating the SHIFTTOKEN signal, each multiplexer 2032(i) is enabled to couple the signalat its other data input terminal, namely, the TR(i) signal output by itsrespective token register stage 2030(i), to the stage's input terminal.The stage 2030(i) latches the signal at the next tick of the ACLK (P)signal. Thus, the negated SHIFT TOKEN signal enables the token registerstages 2030(i) to maintain their respective states.

The TR (m-1) output signal from the last token register stage 2030(m-1)in the token register 2030 is coupled to one data input terminal of amultiplexer 2037. The multiplexer 2037 controls the coupling of the TR(m-1) signal through those of gated drivers 2040(m-1) through 2040(0)[generally identified by reference numeral 2040(i)] to those of thechild diagnostic network nodes whose EN (i) signals are asserted. Eachgated driver 2040(i) provides the ATI (C_(i)) address token in to childsignal, which the respective child node receives as the ATI (P) addresstoken in from parent signal on its line 2014. Thus, after the token haspassed through the series of token register stages 2030(i) in diagnosticnetwork node 100, it can be passed to the child nodes whose EN (i)signals are asserted.

The control circuit 2033 also controls several other operations in theaddress token/data control circuit 2000. In particular, the controlcircuit generates a READ FR ST read flag register state signal whichcontrols a multiplexer 2034. The output terminal of multiplexer 2034 isconnected to line 2015 and provides the ADO (P) address data out toparent signal. One data input terminal of multiplexer 2034 is providedby a series of multiplexers generally identified by reference numeral2035(i). Specifically, each multiplexer 2035(i) receives at one datainput terminal the EN (i) signal from an associated flag 106(C_(i)) inflag register 106, and at another data input terminal the signal fromthe next multiplexer 2035(i+1). The second data input terminal of thelast multiplexer 2035(m-1) is provided by an address data combiningcircuit 2036, which, as described below, receives ADO (C_(i)) signalsfrom child nodes for which corresponding EN (i) signals are asserted,and combines them according to logical operations as selected by theEADO (P) expected address data out signal from line 2012.

If a TR (i) signal from token register 2030 is asserted, thecorresponding multiplexer 2035(i) is enabled to couple the EN (i) signalrepresenting the condition of the associated flag 106(C_(i)) to a datainput terminal of the next multiplexer 2035(i-1) in the series. If theTR (0) signal is asserted, the multiplexer 2035(0) couples the EN (0)signal to a data input terminal of the multiplexer 2034. On the otherhand, if the TR (i) signal is negated, the corresponding multiplexer2035(i) is enabled to couple the signal from the next multiplexer2035(i+1) to the second data input terminal of the next multiplexer2035(i-1) in the series.

Thus, if the token register 2030 is asserting a TR (i) signal, theseries of multiplexers 2035(i) couples the EN (i) enable signal to adata input terminal of the multiplexer 2034. If, however, none of the TR(i) signals is asserted on diagnostic register node 100, the series ofmultiplexers 2035(i) couples the signal from address data combiningcircuit 2036 to the same data input terminal of multiplexer 2034. Ineither case, if the READ FR ST read flag register state signal isasserted, the multiplexer 2034 couples that signal onto line 2015 as theADO (P) signal. The second data input terminal of multiplexer 2034 isconnected directly to the output terminal of the address data combiningcircuit 2036. Accordingly, if the READ FR ST signal is negated, themultiplexer 2034 will couple the output signal from the address datacombining circuit 2036 onto line 2015 as the ADO (P) signal.

The address data combining circuit 2036 receives ADO (C_(i)) signalsfrom the respective child diagnostic network nodes for which the EN (i)enable signals are asserted, combines them according to a logicalfunction identified by the EADO (P) expected address data out signalfrom line 2012, and provides the result to one data input terminal ofmultiplexer 2035(m-1). The address data combining circuit includes twogeneral sections, including an AND section 2041 and an OR section 2042,along with a multiplexer 2043 which is controlled by the EADO (P)signal. If the EADO (P) signal is negated, the multiplexer 2043 couplesan EXP AD NEG expect negated output signal from the OR section 2042 tothe multiplexer 2035(m-1), which will be transmitted through themultiplexers 2035(i) and 2034 as the ADO (P) signal. If the OR section2042 is negating the EXP AD NEG signal, all of the ADO (C_(i)) signals,from the nodes for which the EN (i) signals are asserted, are negated.Accordingly, the conditions of the ADO (C_(i)) signals will correspondto the negated condition of the EADO (P) signal.

However, if one of the ADO (C_(i)) signals, from the nodes for which theEN (i) signals are asserted, is asserted, the OR section 2042 willassert the EXP AD NEG expect negated signal. The asserted signal will becoupled to the multiplexer 2035(m-1) and through the multiplexers2035(i) and 2034 as the ADO (P) signal. In that case, the condition ofat least one of the ADO (C_(i)) signals will differ from the negatedcondition of the EADO (P), indicating an error.

On the other hand, if the EADO (P) signal is asserted, the multiplexer2043 couples an EXP AD AST expect asserted output signal from the ANDsection 2041 to the multiplexer 2035(m-1), which will be transmittedthrough the multiplexers 2035(i) and 2034 as the ADO (P) signal. If theAND section 2042 is asserting the EXP AD AST signal, all of the ADO(C_(i)) signals, from the nodes for which the EN (i) signals areasserted, are asserted. Accordingly, the conditions of those ADO (C_(i))signals will correspond to the asserted condition of the EADO (P)signal.

However, if one of the ADO (C_(i)) signals, from the nodes for which theEN (i) signals are asserted, is negated, the AND section 2041 willnegate the EXP AD AST expect asserted signal. The negated signal will becoupled to the multiplexer 2035(m-1) and through the multiplexers2035(i) and 2034 as the ADO (P) signal. In that case, the condition ofat least one of the ADO (C_(i)) signals will differ from the assertedcondition of the EADO (P), indicating an error.

The AND section 2041 of address data combine circuit 2036 includes anAND gate 2043 which receives input signals from a series of OR gates,generally identified by reference numeral 2045(i). Each OR gate 2045(i)receives at one input terminal an ADO (C_(i)) address data out signalfrom a child diagnostic network node. At its other input terminal, theOR gate 2045(i) receives the complement of the EN (i) signal, asgenerated by an inverter, generally identified by reference numeral2046(i). Accordingly, if an EN (i) signal is not asserted, the inverter2046(i) energizes the OR gate 2045(i) to enable the corresponding inputterminal of AND gate 2044.

On the other hand, if the EN (i) signal is asserted, the inverter2046(i) disables that input terminal of the respective OR gate 2045(i).Thus, the condition of the OR gate 2045(i) is controlled by thecondition of the ADO (C_(i)) signal. If the ADO (C_(i)) signal isasserted, the associated OR gate 2045(i) will be energized to energizethe respective input terminal of the AND gate 2044. However, if an ADO(C_(i)) signal is negated, the associated OR gate 2045(i) will bede-energized to, in turn, disable the AND gate 2044. Thus, if all of theADO (C_(i)) signals from the child diangostic network nodes, for whichEN (i) signals are asserted, are asserted, the EXP AD AST expectasserted signal will be asserted. However, if one of the ADO (C_(i))signals from the child diangostic network nodes, for which EN (i)signals are asserted, is negated, the AND gate 2044 will be disabled andthe EXP AD AST expect asserted signal will be negated.

The OR section 2042 of address data combine circuit 2036 includes an ORgate 2050 which receives input signals from a series of AND gates,generally identified by reference numeral 2051(i). Each AND gate 2051(i)receives ,at one input terminal an ADO (C_(i)) address data out signalfrom a child diagnostic network node. At its other input terminal, theAND gate 2051(i) receives the EN (i) signal from the flags 106(C_(i)) offlag register 106. Accordingly, if an EN (i) signal is asserted, thecorresponding input terminal of AND gate 2051(i) is enabled. On theother hand, if the EN (i) signal is negated, the AND gate 2051(i) isdisabled.

Thus, the condition of the AND gates 2051(i) enabled by the asserted EN(i) signals is controlled by the condition of the ADO (C_(i)) signal. Ifthe ADO (C_(i)) signal is negated, the associated AND gate 2051(i) willbe de-energized to disable the respective input terminal of the OR gate2050. If all ADO (C_(i)) signals, for which EN (i) signals are asserted,are negated, the OR gate 2050 will be de-energized to negate the EXP ADNEG expect negated signal. However, if an ADO (C_(i)) signal, for whichan EN (i) signal is asserted, is negated, the associated AND gate2051(i) will be energized to, in turn, energize the OR gate 2050 andassert the EXP AD NEG signal. Thus, if all of the ADO (C_(i)) signalsfrom the child diagostic network nodes, for which EN (i) signals areasserted, are negated, the EXP AD NEG expect negated signal will benegated. However, if one of the ADO (C_(i)) signals from the childdiagostic network nodes, for which EN (i) signals are asserted, isasserted, the OR gate 2050 will be energized and the EXP AD NEG expectNEGATED signal will be asserted.

As noted above, multiplexer 2037 controls the coupling of the TR (m-1)signal from token register stage 2030(m-1) to an input terminal of eachof gated drivers 2040(i), and those of the EN (i) signals that areasserted enables their respective gated drivers 2040(i) to, in turn,couple the TR (m-1) signal to their respective the child diagnosticnetwork nodes. The TR (m-1) signal is coupled to one data input terminalof the multiplexer 2037. The multiplexer's other data input terminal isconnected to line 2014 to receive the ATI (P) signal. The multiplexer2037 is controlled by the READ FR ST read flag register state signalfrom the control circuit 2033. If the READ FR ST signal is asserted, themultiplexer 2037 couples the TR (m-1) signal to the input terminals ofgated drivers 2040(i) and if the READ FR ST signal is negated themultiplexer 2037 couples the ATI (P) signal thereto.

The diagnostic network node 100 also includes several drivers 2052through 2055 for transmitting several to all of its child nodes. Inparticular, drivers 2052 through 2055 transmit, respectively, the ACLK(P) signal from line 2010, the AMS (P) signal from line 2011, the EADO(P) from line 2012 and the ADI (P) from line 2014, to all of its childnodes as the ACLK (C_(i)), AMS (C_(i)), EADO (C_(i)) and ADI (C_(i))signals. The diagnostic network node 100 effectively broadcasts thesignals, without being gated or controlled by the EN (i) enable signal,to all of its child nodes.

The control circuit 2033 also provides several additional signals forcontrolling the operation of the circuitry depicted on FIG. 13B-1. ARESET TOKEN REG signal enables all of the token register stages 2030(i)of the token register 2030 to be cleared, or reset, to a predeterminedstate. When the stages 2030(i) are reset, all of the TR (i) tokenregister signals are negated. In addition, a RESET FLAG REG signalenables all of the flags 106(C_(i)) to be conditioned to a known state.In one particular embodiment, the flag 106(C₀) is conditioned to a setstate, and the other flags 106(C₁) through 106(C_(m-1)) are cleared.

In that case, the diagnostic processor can determine the configurationof the diagnostic network nodes 100(h,p,r-1) in the diagnostic networkby, after enabling the control circuits 2033 to assert the RESET FLAGREG signal, to iteratively retrieve the states of the respective flags106(C_(i)) in the various nodes. In that operation, the diagnosticprocessor. 101 can control sequencing of a token down the diagnosticnetwork, and if the ADO (P) signal is asserted the diagnostic processorcan determine from that that the location of the token in a tokenregister 2030 identifies the first flag 106(C₀) in a node. As the tokenis sequenced through the token register 2030, the ADO (P) signal will benegated. The token will then be transmitted to the child node connectedto the bus 104(C1) and when it is received in the first token stage2030(0) in the token register 2030 therein, the ADI (P) will again beasserted. Thus, the diagnostic processor can determine the number ofstages in the flag register 106 by determining the number of stepsrequired in the sequence between assertions of the ADO (P) signal.

As noted above, the control circuit 2033 generates the SHIFT TOKEN, READFR ST, RESET TOKEN REG and RESET FLAG REG signals to control the othercircuit elements depicted on FIG. 13B-1. In one embodiment, the controlcircuit 2033 is a state machine controlled by the AMS (P) address modeselect from parent signal and the ACLK (P) address clock signal. Foreach state, the condition of the AMS (P) signal determines a targetstate for the control circuit 2033, and the ticks of the ACLK (P) signaldetermine the timing of the state transition. It will be appreciatedthat, since the AMS (P) and ACLK (P) signals are transmitted to, andreceived by, all diagnostic network nodes 100(h,p,r-1) in parallel, thecontrol circuits 2033 in all of the nodes will be controlled in paralleland will be in the same state at the same time. The various states andstate transitions, and the conditions of the signals generated by acontrol circuit 2033 in each state, are depicted in FIG. 13B-2.

With reference to FIG. 13B-2, the control circuit is initially in areset state, as represented by the box of the same label. In that state,as shown in the Signal Condition/State Table on FIG. 13B-2, the controlcircuit 2033 asserts the RESET TOKEN REG and RESET FLAG REG signals toreset the token register 2030 and flag register 106 as described above.While the diagnostic processor 101 maintains the AMS (P) signalasserted, the control circuit 2033 remains in the reset state.

If the diagnostic processor 101 negates the AMS (P) signal while thecontrol circuit 2033 is in the reset state, the control circuitsequences to a "clear token" state, as represented by the box of thesame label. As noted above, the state transition occurs at the next tickof the ACLK (P) signal after negation of the AMS (P) signal. In theclear token state, the control circuit 2033 asserts the RESET TOKEN REGsignal, to reset the stages 2030(i) of the token register 2030, andmaintains the other signals negated. If the diagnostic processorre-asserts the AMS (P) signal, at the next tick of the ACLK (P) signalthe control circuit 2033 returns to the reset state. Otherwise, if thediagnostic processor maintains the AMS (P) signal negated at the nexttick of the ACLK (P) signal, the control circuit 2033 sequences to ashift token state. The control circuit's shift token state isrepresented by a box on FIG. 13B-2 of the same name. If the diagnosticprocessor 101 thereafter maintains the AMS (P) signal negated atsuccessive ticks of the ACLK (P) signal, the control circuit 2033remains in the shift token state.

In the shift token state, the control circuit 2033 asserts the SHIFTTOKEN signal and the READ FR ST read flag register state signal. Asnoted above, while the SHIFT TOKEN signal is asserted, each multiplexer2032(i+1) couples the TR (i) signal from the preceding token registerstage 2030(i) to be latched in its stage 2030(i+1) at successive ticksof the ACLK (P) signal. Thus, if a token register stage 2030(i) is in acondition indicating that it has a token, or if stage 2030(0) receivesthe token from the parent node or the diagnostic processor 101, whilethe SHIFT TOKEN signal is asserted at successive ticks of the ACLK (P)signal the token shifts through the succeeding stages 2030(i) and outthe gated drivers 2040(i) associated with asserted EN (0) enablesignals. In addition, the READ FR ST signal enables the multiplexer 2034to couple the signal from the multiplexer series 2035(i) as the ADO (P)address data out to parent signal to the parent diagnostic network node.

On the other hand, if, while the control circuit 2033 is in the shifttoken state, the diagnostic processor 101 asserts the AMS (P) signal,the control circuit 2033 sequences to a read flag register state at thenext tick of the ACLK (P) address clock from parent signal. In the readflag register state the control circuit asserts only the READ FR ST readflag register state signal, which, as noted above, enables themultiplexer 2034 to couple the signal from the multiplexer series2035(i) as the ADO (P) address data out to parent signal to the parentdiagnostic network node. If the diagnostic processor 101 maintains theAMS (P) signal in an asserted condition, the control circuit 2033returns to the clear token state at the next tick of the ACLK (P)signal. On the other hand, if the diagnostic processor 101 negates theAMS (P) signal while the control circuit 2033 is in the read flagregister state, the control circuit 2033 returns to the shift tokenstate.

3. Test Data Control Portion 2001

The test data control portion 2001 will be described in connection withFIG. 13C. With reference to FIG. 13C, the test data control portionincludes three general sections. One section transmits several signalsreceived from the parent diagnostic network node, or the diagnosticprocessor 101, directly to the various child diagnostic nodes connectedthereto. In particular, the test data control portion 2001 receives theTDI (P) test data in from parent signal on line 2022 and transmits it toall of the child diagnostic network nodes in parallel through drivers2060(0) through 2060(m-1) [generally identified by reference numeral2060(i)] to all of the child nodes connected to the respective buses110(C_(i)). In addition, the test data control portion 2001 receives theETDO (P) expected test data out signal on line 2023 and couplestransmits it to all of the child diagnostic network nodes in parallelthrouh drivers 2061(0) through 2061(m-1) [generally identified byreference numeral 2061(i)] to all of the child nodes connected to therespective buses 110(C_(i)).

A second section gates several other signals received from the parentdiagnostic network node, or the diagnostic processor 101, to those childnodes whose EN (i) enable signals are asserted. In particular, the testdata control portion 2001 receives the TCLK (P) test clock from parentsignal on line 2020 and transmits it through those of gated drivers2062(0) through 2062(m-1) [generally identified by reference numeral2062(i)] associated with the asserted EN (i) signals. Similarly, thetest data control portion 2001 receives the TMS (P) test mode selectfrom parent signal on line 2020 and transmits it through those of gateddrivers 2063(0) through 2063(m-1) [generally identified by referencenumeral 2063(i)] associated with the asserted EN (i) signals.

Finally, the test data control portion includes a test data combiningcircuit 2064 which receives TDO (C_(i)) signals from child nodes forwhich corresponding EN (i) signals are asserted, and combines themaccording to logical operations as selected by the ETDO (P) expectedtest data out signal from line 2023. The structure and operation of thetest data combining circuit 2064 is generally similar to the addressdata combining circuit 2036 described above.

The test data combining circuit 2064 receives TDO (C_(i)) signals fromthe respective child diagnostic network nodes for which the EN (i)enable signals are asserted, combines them according to a logicalfunction identified by the ETDO (P) expected test data out signal fromline 2023, and provides the result to one data input terminal ofmultiplexer 2035(m-1). The test data combining circuit includes twogeneral sections, including an AND section 2071 and an OR section 2072,along with a multiplexer 2073 which is controlled by the ETDO (P)signal. If the ETDO (P) signal is negated, the multiplexer 2073 couplesan EXP TD NEG expect negated output signal from the OR section 2072 as aCOMB TD OUT combined test data out signal to one input terminal of amultiplexer 2082. If the OR section 2072 is negating the EXP TD NEGsignal, all of the TDO (C_(i)) signals, from the nodes for which the EN(i) signals are asserted, are negated. Accordingly, the conditions ofthe TDO (C_(i)) signals will correspond to the negated condition of theETDO (P) signal.

However, if one of the TDO (C_(i)) signals, from the nodes for which theEN (i) signals are asserted, is asserted, the OR section 2072 willassert the EXP TD NEG expect negated signal. The asserted signal will becoupled to the multiplexer 2035(m-1) and through the multiplexers2035(i) and 2034 as the TDO (P) signal. In that case, the condition ofat least one of the TDO (C_(i)) signals will differ from the negatedcondition of the ETDO (P), indicating an error.

On the other hand, if the ETDO (P) signal is asserted, the multiplexer2073 couples an EXP TD AST expect asserted output signal from the ANDsection 2071 to the multiplexer 2035(m-1), which will be transmittedthrough the multiplexers 2035(i) and 2034 as the TDO (P) signal. If theAND section 2072 is asserting the EXP TD AST signal, all of the TDO(C_(i)) signals, from the nodes for which the EN (i) signals areasserted, are asserted. Accordingly, the conditions of those TDO (C_(i))signals will correspond to the asserted condition of the ETDO (P)signal.

However, if one of the TDO (C_(i)) signals, from the nodes for which theEN (i) signals are asserted, is negated, the AND section 2071 willnegate the EXP TD AST expect asserted signal. The negated signal will becoupled to the multiplexer 2035(m-1) and through the multiplexers2035(i) and 2034 as the TDO (P) signal. In that case, the condition ofat least one of the TDO (C_(i)) signals will differ from the assertedcondition of the ETDO (P), indicating an error.

The AND section 2071 of test data combine circuit 2064 includes an ANDgate 2073 which receives input signals from a series of OR gates,generally identified by reference numeral 2075(i). Each OR gate 2075(i)receives at one input terminal an TDO (C_(i)) test data out signal froma child diagnostic network node. At its other input terminal, the ORgate 2075(i) receives the complement of the EN (i) signal, as generatedby an inverter, generally identified by reference numeral 2076(i).Accordingly, if an EN (i) signal is not asserted, the inverter 2076(i)energizes the OR gate 2075(i) to enable the corresponding input terminalof AND gate 2074.

On the other hand, if the EN (i) signal is asserted, the inverter2076(i) disables that input terminal of the respective OR gate 2075(i).Thus, the condition of the OR gate 2075(i) is controlled by thecondition of the TDO (C_(i)) signal. If the TDO (C_(i)) signal isasserted, the associated OR gate 2075(i) will be energized to energizethe respective input terminal of the AND gate 2074. However, if an TDO(C_(i)) signal is negated, the associated OR gate 2075(i) will bede-energized to, in turn, disable the AND gate 2074. Thus, if all of theTDO (C_(i)) signals from the child diangostic network nodes, for whichEN (i) signals are asserted, are asserted, the EXP TD AST expectasserted signal will be asserted. However, if one of the TDO (C_(i))signals from the child diangostic network nodes, for which EN (i)signals are asserted, is negated, the AND gate 2074 will be disabled andthe EXP TD AST expect asserted signal will be negated.

The OR section 2072 of test data combine circuit 2064 includes an ORgate 2080 which receives input signals from a series of AND gates,generally identified by reference numeral 2081(i). Each AND gate 2081(i)receives at one input terminal an TDO (C_(i)) test data out signal froma child diagnostic network node. At its other input terminal, the ANDgate 2081(i) receives the EN (i) signal from the flags 106(C_(i)) offlag register 106. Accordingly, if an EN (i) signal is asserted, thecorresponding input terminal of AND gate 2081(i) is enabled. On theother hand, if the EN (i) signal is negated, the AND gate 2081(i) isdisabled.

Thus, the condition of the AND gates 2081(i) enabled by the asserted EN(i) signals is controlled by the condition of the TDO (C_(i)) signal. Ifthe TDO (C_(i)) signal is negated, the associated AND gate 2081(i) willbe de-energized to disable the respective input terminal of the OR gate2080. If all TDO (C_(i)) signals, for which EN (i) signals are asserted,are negated, the OR gate 2080 will be de-energized to negate the EXP TDNEG expect negated signal. However, if an TDO (C_(i)) signal, for whichan EN (i) signal is asserted, is negated, the associated AND gate2081(i) will be energized to, in turn, energize the OR gate 2080 andassert the EXP TD NEG signal. Thus, if all of the TDO (C_(i)) signalsfrom the child diagostic network nodes, for which EN (i) signals areasserted, are negated, the EXP TD NEG expect negated signal will benegated. However, if one of the TDO (C_(i)) signals from the childdiagostic network nodes, for which EN (i) signals are asserted, isasserted, the OR gate 2080 will be energized and the EXP TD NEG expectnegated signal will be asserted.

The multiplexer 2082 determines the source of signals coupled onto line2024 as the TDO (P) test data out to parent signal. An AND gate 2083,controlled by the complements of the EN (i) signals as generated byinverters 2084(0) through 2084(m-1) [generally identified by referencenumeral 2084(i)], asserts a NONE EN none enabled signal if the flags106(C_(i)) are not asserting any of the EN (i) enable signals. If theNONE EN signal is negated, indicating that at least one EN (i) enablesignal is asserted, the multiplexer 2082 couples the COMB TD OUT signalonto line 2024 as the TDO (P) test data out signal. On the other hand,if the NONE EN signal is asserted, the multiplexer 2082 couples the TDI(P) signal received on line 2022 onto line 2024 as the TDO (P) signal.

As noted above, the test data combine portion 2064, particularly the ANDsection 2071 and the OR section 2072, along with multiplexer 2073, issimilar to the address data combine portion 2036 of the addresstoken/data control portion 2000. In addition, it will be recognized thatthe address data combine portion 2036 and the test daa combine portion2064 will be used at different points in time. That is, the address datacombine portion 2036 will be used while the flags 106(C_(i)) are beingconditioned, and the test data combine portion will be used thereafter.Accordingly, in one specific embodiment, the same circuitry is used forboth elements.

The foregoing description has been limited to a specific embodiment ofthis invention. It will be apparent, however, that variations andmodifications may be made to the invention, with the attainment of someor all of the advantages of the invention. Therefore, it is the objectof the appended claims to cover all such variations and modifications ascome within the true spirit and scope of the invention.

What is claimed as new and desired to be secured by Letters Patent ofthe United States is:
 1. A digital computer comprisingA. a plurality ofprocessing elements each performing data processing operations inconnection with commands, at least some of the processing elementsperforming said data processing operations in connection with saidcommands which they receive in control network messages, each processingelement also (i) generating and receiving data transfer messages, eachincluding an address portion containing an address, for transfer toanother processing element as identified by the address and (ii) atleast one of said processing elements further generating said controlnetwork messages and (iii) said at least some processing elementsreceiving said control network messages; B. a communications routercomprising a plurality of router node groups interconnected in a treepattern in a series of levels from a lower leaf level to an upper rootlevel, and each node group in levels above the leaf level including aplurality of router nodes, with router nodes in levels below the rootlevel being connected to a plurality of router nodes in the next higherlevel thereby forming a fat-tree structure, each node receiving datatransfer messages and coupling them to another node or to a processingelement connected thereto as determined by the address in the respectiveaddress portion; and C. a control network comprising a like plurality ofcontrol network node groups interconnected in a like tree pattern in aseries of levels from a lower leaf level to an upper physical rootlevel, each control network node group below the upper root levelreceiving control network messages from a processing element or alower-level control network node group and generating a control networkmessage in response thereto for transmission to a higher-level controlnetwork node group, and receiving control network messages from a higherlevel control network node group and generating control messages inresponse thereto for transmission to lower-level control network nodegroups, the control network node group at the root level generatingcontrol network messages for transmission to the lower level controlnetwork node groups in response to control network messages receivedtherefrom.
 2. A digital computer as defined in claim 1 in which eachcontrol network node group in the control network is associated with arouter node group in the communications router, each control networknode group providing a control signal for controlling a selectedoperation of the associated router node group.
 3. A digital computer asdefined in claim 2 in which each respective control signal provided by asaid control network node group is provided in response to controlinformation provided by at least one of said processing elements.
 4. Adigital computer as defined in claim 3 in which said control informationis provided in a said control network message.
 5. A digital computer asdefined in claim 4 in which said control network node groups selectivelyoperate in a first mode and a second mode, the control signal enablingsaid router node groups to operate in one of said first mode or saidsecond mode, the router node groups when operating in said second modetransferring said data transfer messages to respective router nodegroups in said next lower levels thereby enabling said communicationsrouter to quickly transfer data transfer messages to said processingelements.